[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:31 dist-test-slave-btw8 krb5kdc[802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:31 dist-test-slave-btw8 krb5kdc[802](info): set up 2 sockets
Aug 07 19:43:31 dist-test-slave-btw8 krb5kdc[802](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 19:43:32.339342 795 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:43:32 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:43:32.590541 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:43593
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:43593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:32.963269 815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:32.963769 815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:32.963950 815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:32.972225 815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:32.972506 815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:32.972705 815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:32.972862 815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:32.972950 815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:32.984122 815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:43593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:43593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:32.985603 815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:32.987118 815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:33.000378 820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:33.000510 821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:33.000483 823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:33.001566 815 server_base.cc:1034] running on GCE node
I20260807 19:43:33.002483 815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:33.004172 815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:33.005460 815 hybrid_clock.cc:648] HybridClock initialized: now 1786131813005422 us; error 39 us; skew 500 ppm
I20260807 19:43:33.005950 815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:33 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:33.013283 815 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:43:33.015101 815 webserver.cc:460] Webserver started at http://127.0.198.254:39395/ using document root <none> and password file <none>
I20260807 19:43:33.016002 815 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:33.016158 815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:33.016538 815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:33.020140 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "7a731f1572274414ba462056f85026e5"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "fcd7f3db3a75ea679395c754da55c6da"
server_key_iv: "58eedb2f7f890f7ac91855a5f198a40e"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.021088 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "7a731f1572274414ba462056f85026e5"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "fcd7f3db3a75ea679395c754da55c6da"
server_key_iv: "58eedb2f7f890f7ac91855a5f198a40e"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.027012 815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:43:33.031065 832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:33.032935 815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:43:33.033176 815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "7a731f1572274414ba462056f85026e5"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "fcd7f3db3a75ea679395c754da55c6da"
server_key_iv: "58eedb2f7f890f7ac91855a5f198a40e"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.033407 815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:33.073263 815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:33.078883 815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:33.079288 815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:33.101131 815 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:43593
I20260807 19:43:33.101168 884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:43593 every 8 connection(s)
I20260807 19:43:33.102674 815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:33.106563 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 815
I20260807 19:43:33.106938 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:43:33.107468 885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:33.109704 795 external_mini_cluster.cc:1428] Setting key d6fdd9f1105fc04db9bfed7ef07fecf0
I20260807 19:43:33.121969 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5: Bootstrap starting.
I20260807 19:43:33.125872 885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:33.127677 885 log.cc:824] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:33.131695 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5: No bootstrap required, opened a new log
I20260807 19:43:33.142584 885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } }
I20260807 19:43:33.143105 885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:33.143246 885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a731f1572274414ba462056f85026e5, State: Initialized, Role: FOLLOWER
I20260807 19:43:33.144009 885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } }
I20260807 19:43:33.144304 885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:33.144477 885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:33.144632 885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 19:43:33 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:33.147921 885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } }
I20260807 19:43:33.148767 885 leader_election.cc:302] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a731f1572274414ba462056f85026e5; no voters:
I20260807 19:43:33.149778 885 leader_election.cc:288] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:33.149982 890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:33.151070 890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [term 1 LEADER]: Becoming Leader. State: Replica: 7a731f1572274414ba462056f85026e5, State: Running, Role: LEADER
I20260807 19:43:33.151893 890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } }
I20260807 19:43:33.153606 885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:33.155285 892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a731f1572274414ba462056f85026e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } } }
I20260807 19:43:33.155750 892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:33.158368 891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a731f1572274414ba462056f85026e5. Latest consensus state: current_term: 1 leader_uuid: "7a731f1572274414ba462056f85026e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a731f1572274414ba462056f85026e5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43593 } } }
I20260807 19:43:33.158825 891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:33.161490 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:33.115238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.0.198.254:43593)
0807 19:43:33.116290 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:33.116309 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:33.117854 (+ 1545us) server_negotiation.cc:407] Connection header received
0807 19:43:33.128751 (+ 10897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:33.128779 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:33.129394 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:33.129925 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:33.133186 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.135628 (+ 2442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.137691 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.139089 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.139375 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:33.146463 (+ 7088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:33.146480 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:33.146505 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:33.146582 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:33.152620 (+ 6038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:33.154003 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:33.154014 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:33.154022 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:33.154321 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:33.155121 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:33.155129 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:33.155134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:33.156213 (+ 1079us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:33.156702 (+ 489us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:33.159907 (+ 3205us) server_negotiation.cc:299] Negotiation successful
0807 19:43:33.160340 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":429,"threads_started":1}
I20260807 19:43:33.161687 895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:33.172735 895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:33.185369 895 catalog_manager.cc:1347] Generated new cluster ID: a92c3efe48c54135bc465894d8089e5d
I20260807 19:43:33.185544 895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:33.204237 895 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:33.205850 895 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:33.228440 895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a731f1572274414ba462056f85026e5: Generated new TSK 0
I20260807 19:43:33.229605 895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:43:33.300904 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:33.678433 911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:33.678802 911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:33.678978 911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:33.686643 911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:33.686839 911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:33.687021 911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:43:33.697916 911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:33.699429 911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:33.700829 911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:33.715342 917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:33.715397 919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:33.715441 916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:33.716969 911 server_base.cc:1034] running on GCE node
I20260807 19:43:33.717844 911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:33.718989 911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:33.720249 911 hybrid_clock.cc:648] HybridClock initialized: now 1786131813720157 us; error 84 us; skew 500 ppm
I20260807 19:43:33.720726 911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:33 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:33.724635 911 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:43:33.726302 911 webserver.cc:460] Webserver started at http://127.0.198.193:39063/ using document root <none> and password file <none>
I20260807 19:43:33.727088 911 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:33.727205 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:33.727588 911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:33.731420 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "ba193abfe86f44c0bc91727434dffd6b"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "3d260e5b43c99b7661b201ee01eef8d7"
server_key_iv: "1598a2a70234f18ce2eb3bfbd72d0bd9"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.732405 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "ba193abfe86f44c0bc91727434dffd6b"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "3d260e5b43c99b7661b201ee01eef8d7"
server_key_iv: "1598a2a70234f18ce2eb3bfbd72d0bd9"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.739178 911 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:43:33.743703 926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:33.745963 911 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:43:33.746196 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "ba193abfe86f44c0bc91727434dffd6b"
format_stamp: "Formatted at 2026-08-07 19:43:33 on dist-test-slave-btw8"
server_key: "3d260e5b43c99b7661b201ee01eef8d7"
server_key_iv: "1598a2a70234f18ce2eb3bfbd72d0bd9"
server_key_version: "encryptionkey@0"
I20260807 19:43:33.746460 911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:33.762555 911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:33.767876 911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:33.768234 911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:33.769431 911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:33.771062 911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:33.771176 911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:33.771386 911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:33.771473 911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 19:43:33.813681 911 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:33011
I20260807 19:43:33.813750 1046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:33011 every 8 connection(s)
I20260807 19:43:33.815277 911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:33.817409 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 911
I20260807 19:43:33.817802 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:33.818802 795 external_mini_cluster.cc:1428] Setting key 170c247169e3b15c4b982bc42bc4d2fd
Aug 07 19:43:33 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:43:33.841563 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:33.818597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:48917 (local address 127.0.198.254:43593)
0807 19:43:33.819959 (+ 1362us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:33.819966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:33.820903 (+ 937us) server_negotiation.cc:407] Connection header received
0807 19:43:33.823071 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:33.823078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:33.823182 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:33.823331 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:33.825156 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.826014 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.827955 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.828415 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.828586 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:33.833627 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:33.833634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:33.833637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:33.833681 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:33.836786 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:33.838184 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:33.838190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:33.838192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:33.838359 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:33.838972 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:33.838977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:33.838980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:33.839586 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:33.839815 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:33.840858 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0807 19:43:33.841149 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":1076,"threads_started":1}
I20260807 19:43:33.842772 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:33.819309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43593 (local address 127.0.198.193:48917)
0807 19:43:33.820709 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:33.820761 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:33.822755 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:33.823642 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:33.823653 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:33.824274 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:33.824936 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:33.824956 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.826226 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.826231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:33.827762 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:33.827777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:33.828706 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:33.828714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:33.828998 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:33.829926 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:33.829965 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:33.833266 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:33.837034 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:33.837045 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:33.837063 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:33.837990 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:33.838506 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:33.838515 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:33.838524 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:33.838809 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:33.839747 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:33.839758 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:33.840311 (+ 553us) client_negotiation.cc:769] Sending connection context
0807 19:43:33.840652 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 19:43:33.841340 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"thread_start_us":838,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:43:33.845018 1047 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43593
I20260807 19:43:33.845451 1047 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:33.846611 1047 heartbeater.cc:511] Master 127.0.198.254:43593 requested a full tablet report, sending...
I20260807 19:43:33.849509 850 ts_manager.cc:194] Registered new tserver with Master: ba193abfe86f44c0bc91727434dffd6b (127.0.198.193:33011)
I20260807 19:43:33.851536 850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:48917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:43:33.875833 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:34.272878 1053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:34.273200 1053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:34.273301 1053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:34.280782 1053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:34.280913 1053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:34.281035 1053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:43:34.291548 1053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:34.292901 1053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:34.294222 1053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:34.311858 1059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:34.311774 1060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:34.311812 1062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:34.312835 1053 server_base.cc:1034] running on GCE node
I20260807 19:43:34.313598 1053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:34.314998 1053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:34.316294 1053 hybrid_clock.cc:648] HybridClock initialized: now 1786131814316249 us; error 35 us; skew 500 ppm
I20260807 19:43:34.316785 1053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:34 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:34.320811 1053 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:43:34.322531 1053 webserver.cc:460] Webserver started at http://127.0.198.194:36525/ using document root <none> and password file <none>
I20260807 19:43:34.323311 1053 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:34.323523 1053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:34.323884 1053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:34.327097 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "011360a96e734c909eb9a13aaf393073"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "1328073f2f38b3ae273c6b3fdb11fd8d"
server_key_iv: "0ccc969b7b8343add329ff23edd868e4"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.328068 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "011360a96e734c909eb9a13aaf393073"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "1328073f2f38b3ae273c6b3fdb11fd8d"
server_key_iv: "0ccc969b7b8343add329ff23edd868e4"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.334538 1053 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 19:43:34.339131 1069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:34.341084 1053 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:43:34.341342 1053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "011360a96e734c909eb9a13aaf393073"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "1328073f2f38b3ae273c6b3fdb11fd8d"
server_key_iv: "0ccc969b7b8343add329ff23edd868e4"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.341687 1053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:34.369226 1053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:34.374825 1053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:34.375339 1053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:34.376809 1053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:34.378468 1053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:34.378578 1053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:34.378719 1053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:34.378798 1053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:34.419037 1053 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:34231
I20260807 19:43:34.419090 1184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:34231 every 8 connection(s)
I20260807 19:43:34.420773 1053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:34.422258 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1053
I20260807 19:43:34.422505 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:34.423148 795 external_mini_cluster.cc:1428] Setting key 39022d15051299840d164115f13bd7a7
Aug 07 19:43:34 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:34.447602 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:34.424281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:38705 (local address 127.0.198.254:43593)
0807 19:43:34.424865 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:34.424873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:34.426022 (+ 1149us) server_negotiation.cc:407] Connection header received
0807 19:43:34.428192 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:34.428201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:34.428308 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:34.428449 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:34.430073 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:34.431422 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:34.433368 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:34.434281 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:34.434466 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:34.439746 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:34.439753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:34.439756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:34.439799 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:34.442938 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:34.444303 (+ 1365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:34.444308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:34.444310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:34.444475 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:34.445134 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:34.445138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:34.445140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:34.445559 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:34.445702 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:34.446884 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0807 19:43:34.447218 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":251,"threads_started":1}
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
I20260807 19:43:34.449052 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:34.424908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43593 (local address 127.0.198.194:38705)
0807 19:43:34.425827 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:34.425881 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:34.427897 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:34.428707 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:34.428715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:34.429266 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:34.429876 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:34.429894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:34.431594 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:34.431598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:34.433178 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:34.433193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:34.434629 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:34.434635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:34.434894 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:34.435853 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:34.435878 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:34.439309 (+ 3431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:34.443167 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:34.443182 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:34.443206 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:34.444120 (+ 914us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:34.444623 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:34.444632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:34.444638 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:34.444973 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:34.445742 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:34.445754 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:34.446355 (+ 601us) client_negotiation.cc:769] Sending connection context
0807 19:43:34.446758 (+ 403us) client_negotiation.cc:240] Negotiation successful
0807 19:43:34.447526 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":415,"threads_started":1}
I20260807 19:43:34.450966 1185 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
I20260807 19:43:34.451402 1185 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:34.452421 1185 heartbeater.cc:511] Master 127.0.198.254:43593 requested a full tablet report, sending...
I20260807 19:43:34.454535 850 ts_manager.cc:194] Registered new tserver with Master: 011360a96e734c909eb9a13aaf393073 (127.0.198.194:34231)
I20260807 19:43:34.456235 850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:38705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:43:34.483590 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:34.855510 1047 heartbeater.cc:503] Master 127.0.198.254:43593 was elected leader, sending a full tablet report...
W20260807 19:43:34.869876 1192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:34.870237 1192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:34.870430 1192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:34.878396 1192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:34.878556 1192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:34.878772 1192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:43:34.889516 1192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:34.891080 1192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:34.892695 1192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:34.908420 1198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:34.908529 1200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:34.908547 1197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:34.909871 1192 server_base.cc:1034] running on GCE node
I20260807 19:43:34.910526 1192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:34.911654 1192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:34.912876 1192 hybrid_clock.cc:648] HybridClock initialized: now 1786131814912835 us; error 30 us; skew 500 ppm
I20260807 19:43:34.913229 1192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:34 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:34.917315 1192 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:43:34.919550 1192 webserver.cc:460] Webserver started at http://127.0.198.195:40699/ using document root <none> and password file <none>
I20260807 19:43:34.920578 1192 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:34.920787 1192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:34.921195 1192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:34.924980 1192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "1c2edf5f0a7f422b835a9c726d96b731"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "ff7cba2c29fd8883a50e3091f989080d"
server_key_iv: "9fd85939c9127e8c5f05847a73062d1d"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.926100 1192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "1c2edf5f0a7f422b835a9c726d96b731"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "ff7cba2c29fd8883a50e3091f989080d"
server_key_iv: "9fd85939c9127e8c5f05847a73062d1d"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.932973 1192 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:43:34.938123 1207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:34.940263 1192 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:43:34.940524 1192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "1c2edf5f0a7f422b835a9c726d96b731"
format_stamp: "Formatted at 2026-08-07 19:43:34 on dist-test-slave-btw8"
server_key: "ff7cba2c29fd8883a50e3091f989080d"
server_key_iv: "9fd85939c9127e8c5f05847a73062d1d"
server_key_version: "encryptionkey@0"
I20260807 19:43:34.940799 1192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:34.972877 1192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:34.978427 1192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:34.978825 1192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:34.980207 1192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:34.981997 1192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:34.982131 1192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:34.982312 1192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:34.982396 1192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:35.024842 1192 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:44857
I20260807 19:43:35.024951 1322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:44857 every 8 connection(s)
I20260807 19:43:35.026391 1192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:35.030025 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1192
I20260807 19:43:35.030253 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:35.031060 795 external_mini_cluster.cc:1428] Setting key d556900603d7a2a98f241abbd3a32227
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:35.054365 1325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.029668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:39925 (local address 127.0.198.254:43593)
0807 19:43:35.030332 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.030342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.031724 (+ 1382us) server_negotiation.cc:407] Connection header received
0807 19:43:35.034628 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.034633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.034712 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.034820 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.037128 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.038347 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.040580 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.041170 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.041302 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.045949 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.045956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.045959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.046000 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.050146 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.051459 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.051463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.051465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.051656 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.052215 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.052220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.052222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.052556 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.052744 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.053569 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.053851 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":318,"threads_started":1}
I20260807 19:43:35.055627 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.030286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43593 (local address 127.0.198.195:39925)
0807 19:43:35.031465 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:35.031522 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:35.034259 (+ 2737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:35.035197 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:35.035212 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:35.036207 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:35.036912 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:35.036935 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.038644 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.038649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:35.040341 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:35.040362 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.041473 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.041479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:35.041762 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.042650 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:35.042681 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:35.045620 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:35.050320 (+ 4700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:35.050329 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:35.050346 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:35.051236 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:35.051790 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:35.051796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:35.051802 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:35.052040 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:35.052713 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:35.052723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:35.053145 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 19:43:35.053463 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:43:35.054174 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":447,"threads_started":1}
I20260807 19:43:35.057564 1323 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43593
I20260807 19:43:35.057972 1323 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:35.058911 1323 heartbeater.cc:511] Master 127.0.198.254:43593 requested a full tablet report, sending...
I20260807 19:43:35.060968 850 ts_manager.cc:194] Registered new tserver with Master: 1c2edf5f0a7f422b835a9c726d96b731 (127.0.198.195:44857)
I20260807 19:43:35.061883 850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:39925
I20260807 19:43:35.062620 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:35.125218 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.105212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.0.198.193:33011)
0807 19:43:35.106046 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.106056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.106088 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:43:35.106160 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.106169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.106456 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.106619 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.108285 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.109997 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.111626 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.112352 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.112533 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.118215 (+ 5682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.118238 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.118256 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.118315 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.121767 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.122673 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.122711 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.122713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.122865 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.123308 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.123312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.123314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.123846 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.124100 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.124607 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.124865 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":489,"threads_started":1}
W20260807 19:43:35.126828 954 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49748
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:35.186426 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.170123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.0.198.193:33011)
0807 19:43:35.170419 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.170428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.170842 (+ 414us) server_negotiation.cc:407] Connection header received
0807 19:43:35.171138 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.171143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.171215 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.171377 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.172900 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.173598 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.174892 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.175513 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.175664 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.180395 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.180402 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.180405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.180449 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.183578 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.184379 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.184385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.184390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.184549 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.184949 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.184955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.184958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.185261 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.185516 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.185839 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.186079 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 19:43:35.188355 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 911
I20260807 19:43:35.204282 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.189837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49758 (local address 127.0.198.193:33011)
0807 19:43:35.190055 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.190060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.190098 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:43:35.190323 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.190327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.190394 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.190492 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.192195 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.192973 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.194526 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.195064 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.195211 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.197817 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.197823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.197827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.197868 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.201208 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.202045 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.202050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.202052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.202248 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.202646 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.202650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.202652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.202958 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.203190 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.203633 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.203827 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 19:43:35.205219 1041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:35.295915 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 911
I20260807 19:43:35.315743 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1053
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:43:35.336091 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.316759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.0.198.194:34231)
0807 19:43:35.317397 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.317404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.317431 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:43:35.317496 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.317502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.317735 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.317871 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.319540 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.320797 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.322377 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.323150 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.323284 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.328532 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.328549 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.328632 (+ 83us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.328704 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.332254 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.333234 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.333244 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.333246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.333396 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.334014 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.334019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.334022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.334569 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.334883 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.335240 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.335772 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":363,"threads_started":1}
I20260807 19:43:35.337988 1180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:35.427187 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1053
I20260807 19:43:35.447645 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1192
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:43:35.468425 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:35.448899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.0.198.195:44857)
0807 19:43:35.449578 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:35.449585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:35.449617 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:43:35.449701 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:35.449708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:35.449934 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:35.450073 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:35.451645 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.452723 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.454245 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:35.454884 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:35.455089 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:35.460430 (+ 5341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:35.460450 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:35.460468 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:35.460520 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:35.464522 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.465601 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.465613 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.465615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.465768 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:35.466204 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:35.466209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:35.466211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:35.466731 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:35.467010 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:35.467700 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 19:43:35.468045 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":365,"threads_started":1}
I20260807 19:43:35.469930 1318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:35.558681 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1192
I20260807 19:43:35.580389 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 815
I20260807 19:43:35.581840 880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:35.693360 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 815
2026-08-07T19:43:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4579 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[1346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[1346](info): set up 2 sockets
Aug 07 19:43:35 dist-test-slave-btw8 krb5kdc[1346](info): commencing operation
krb5kdc: starting...
W20260807 19:43:36.856563 795 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:43:37 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:43:37.120987 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:41977
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:41977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:37.476085 1359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:37.476424 1359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:37.476588 1359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:37.484349 1359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:37.484480 1359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:37.484568 1359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:37.484656 1359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:37.484751 1359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:37.495585 1359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:41977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:41977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:37.497150 1359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:37.498605 1359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:37.511926 1364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:37.511971 1365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:37.512001 1367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:37.513397 1359 server_base.cc:1034] running on GCE node
I20260807 19:43:37.514463 1359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:37.516444 1359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:37.517777 1359 hybrid_clock.cc:648] HybridClock initialized: now 1786131817517732 us; error 62 us; skew 500 ppm
I20260807 19:43:37.518262 1359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:37 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131817, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:37.522572 1359 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:43:37.524623 1359 webserver.cc:460] Webserver started at http://127.0.198.254:35177/ using document root <none> and password file <none>
I20260807 19:43:37.525504 1359 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:37.525688 1359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:37.526069 1359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:37.529831 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "1b61706c2a074780b9f0cfe526853625"
format_stamp: "Formatted at 2026-08-07 19:43:37 on dist-test-slave-btw8"
server_key: "0dc2bfa90b69bdb4981677080ee84ce4"
server_key_iv: "b870453be30e70468781d1ac66195681"
server_key_version: "encryptionkey@0"
I20260807 19:43:37.531373 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "1b61706c2a074780b9f0cfe526853625"
format_stamp: "Formatted at 2026-08-07 19:43:37 on dist-test-slave-btw8"
server_key: "0dc2bfa90b69bdb4981677080ee84ce4"
server_key_iv: "b870453be30e70468781d1ac66195681"
server_key_version: "encryptionkey@0"
I20260807 19:43:37.538983 1359 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260807 19:43:37.543572 1374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:37.545161 1359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:43:37.545403 1359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "1b61706c2a074780b9f0cfe526853625"
format_stamp: "Formatted at 2026-08-07 19:43:37 on dist-test-slave-btw8"
server_key: "0dc2bfa90b69bdb4981677080ee84ce4"
server_key_iv: "b870453be30e70468781d1ac66195681"
server_key_version: "encryptionkey@0"
I20260807 19:43:37.545653 1359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:37.577252 1359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:37.583791 1359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:37.584229 1359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:37.611297 1359 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:41977
I20260807 19:43:37.611373 1426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:41977 every 8 connection(s)
I20260807 19:43:37.614161 1359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:37.617033 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1359
I20260807 19:43:37.617429 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:43:37.618191 795 external_mini_cluster.cc:1428] Setting key 27e895832143979eb23c5d2224c266ce
I20260807 19:43:37.620522 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:43:37 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:37.639660 1427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625: Bootstrap starting.
I20260807 19:43:37.644722 1427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:37.646255 1427 log.cc:824] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:37.651247 1427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625: No bootstrap required, opened a new log
I20260807 19:43:37.653429 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:37.620676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.0.198.254:41977)
0807 19:43:37.621730 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:37.621749 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:37.621802 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:43:37.624327 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:37.624363 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:37.625017 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:37.625852 (+ 835us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:37.628137 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:37.630602 (+ 2465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:37.633376 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:37.634742 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:37.635022 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:37.641567 (+ 6545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:37.641581 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:37.641609 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:37.641679 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:37.646243 (+ 4564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:37.647530 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:37.647544 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:37.647557 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:37.647897 (+ 340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:37.648594 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:37.648603 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:37.648610 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:37.649573 (+ 963us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:37.649959 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:37.651736 (+ 1777us) server_negotiation.cc:299] Negotiation successful
0807 19:43:37.652267 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":494,"threads_started":1}
I20260807 19:43:37.662674 1427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } }
I20260807 19:43:37.663126 1427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:37.663261 1427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b61706c2a074780b9f0cfe526853625, State: Initialized, Role: FOLLOWER
I20260807 19:43:37.664068 1427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } }
I20260807 19:43:37.664347 1427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:37.664498 1427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:37.664695 1427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:37.667577 1427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } }
I20260807 19:43:37.668190 1427 leader_election.cc:302] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b61706c2a074780b9f0cfe526853625; no voters:
I20260807 19:43:37.668879 1427 leader_election.cc:288] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:37.669044 1432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:37.670282 1432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [term 1 LEADER]: Becoming Leader. State: Replica: 1b61706c2a074780b9f0cfe526853625, State: Running, Role: LEADER
I20260807 19:43:37.670943 1432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } }
I20260807 19:43:37.671217 1427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:37.673962 1434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b61706c2a074780b9f0cfe526853625. Latest consensus state: current_term: 1 leader_uuid: "1b61706c2a074780b9f0cfe526853625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } } }
I20260807 19:43:37.674111 1433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b61706c2a074780b9f0cfe526853625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61706c2a074780b9f0cfe526853625" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41977 } } }
I20260807 19:43:37.674600 1433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:37.674865 1434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:37.677757 1437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:37.684286 1437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:37.694159 1437 catalog_manager.cc:1347] Generated new cluster ID: e57c1dcb15ba4451a3762f3782f3a3af
I20260807 19:43:37.694309 1437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:37.705210 1437 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:37.706691 1437 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:37.720229 1437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b61706c2a074780b9f0cfe526853625: Generated new TSK 0
I20260807 19:43:37.721273 1437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:43:37.787403 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:38.153920 1453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:38.154268 1453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:38.154392 1453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:38.162014 1453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:38.162151 1453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:38.162281 1453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:43:38.173377 1453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:38.174831 1453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:38.176479 1453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:38.192876 1458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:38.192975 1461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:38.193033 1459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:38.194262 1453 server_base.cc:1034] running on GCE node
I20260807 19:43:38.195087 1453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:38.196379 1453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:38.197664 1453 hybrid_clock.cc:648] HybridClock initialized: now 1786131818197617 us; error 42 us; skew 500 ppm
I20260807 19:43:38.198058 1453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:38 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:38.206453 1453 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:43:38.208431 1453 webserver.cc:460] Webserver started at http://127.0.198.193:36711/ using document root <none> and password file <none>
I20260807 19:43:38.209200 1453 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:38.209321 1453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:38.209618 1453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:38.213079 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "4fe5c0aaf1d141be9e082fc76d562ec6"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "c17e65d8bf72134609d55241589493b5"
server_key_iv: "0573dc7e1d04bc59b8fd407993395de3"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.213966 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "4fe5c0aaf1d141be9e082fc76d562ec6"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "c17e65d8bf72134609d55241589493b5"
server_key_iv: "0573dc7e1d04bc59b8fd407993395de3"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.220276 1453 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:43:38.224656 1468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.226339 1453 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 19:43:38.226521 1453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "4fe5c0aaf1d141be9e082fc76d562ec6"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "c17e65d8bf72134609d55241589493b5"
server_key_iv: "0573dc7e1d04bc59b8fd407993395de3"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.226774 1453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:38.242540 1453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:38.247642 1453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:38.247932 1453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:38.248957 1453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:38.250728 1453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:38.250811 1453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.250900 1453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:38.250946 1453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.288161 1453 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:34691
I20260807 19:43:38.288268 1589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:34691 every 8 connection(s)
I20260807 19:43:38.289709 1453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:38.294035 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1453
I20260807 19:43:38.294304 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:38.295038 795 external_mini_cluster.cc:1428] Setting key eb544ff29558396c23ff786b72beb99f
Aug 07 19:43:38 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:38.314896 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:38.292838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:54515 (local address 127.0.198.254:41977)
0807 19:43:38.293380 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:38.293386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:38.294554 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 19:43:38.297476 (+ 2922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:38.297482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:38.297588 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:38.297718 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:38.299615 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.300815 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.302480 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.303227 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.303447 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:38.307583 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:38.307588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:38.307591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:38.307630 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:38.310452 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:38.311834 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:38.311839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:38.311841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:38.311988 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:38.312631 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:38.312635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:38.312638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:38.313011 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:38.313154 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:38.314333 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0807 19:43:38.314593 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":288,"threads_started":1}
I20260807 19:43:38.316644 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:38.293444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:41977 (local address 127.0.198.193:54515)
0807 19:43:38.294345 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:38.294397 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:38.296957 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:38.298005 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:38.298015 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:38.298653 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:38.299336 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:38.299418 (+ 82us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.301008 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.301012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:38.302310 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:38.302323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.303566 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.303571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:38.303827 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:38.304581 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:38.304606 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:38.307279 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:38.310666 (+ 3387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:38.310690 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:38.310714 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:38.311622 (+ 908us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:38.312114 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:38.312121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:38.312128 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:38.312418 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:38.313165 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:38.313177 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:38.313767 (+ 590us) client_negotiation.cc:769] Sending connection context
0807 19:43:38.314135 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 19:43:38.314836 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":400,"threads_started":1}
I20260807 19:43:38.318768 1590 heartbeater.cc:347] Connected to a master server at 127.0.198.254:41977
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:43:38.319209 1590 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:38.320185 1590 heartbeater.cc:511] Master 127.0.198.254:41977 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:43:38.323165 1391 ts_manager.cc:194] Registered new tserver with Master: 4fe5c0aaf1d141be9e082fc76d562ec6 (127.0.198.193:34691)
I20260807 19:43:38.325503 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:54515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:43:38.352665 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:38.708887 1597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:38.709219 1597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:38.709393 1597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:38.717312 1597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:38.717450 1597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:38.717649 1597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:43:38.728451 1597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:38.729914 1597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:38.731534 1597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:38.747676 1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:38.747773 1602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:38.747704 1606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:38.754244 1597 server_base.cc:1034] running on GCE node
I20260807 19:43:38.755229 1597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:38.756537 1597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:38.757814 1597 hybrid_clock.cc:648] HybridClock initialized: now 1786131818757754 us; error 47 us; skew 500 ppm
I20260807 19:43:38.758258 1597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:38 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:38.762372 1597 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:43:38.764325 1597 webserver.cc:460] Webserver started at http://127.0.198.194:42373/ using document root <none> and password file <none>
I20260807 19:43:38.765141 1597 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:38.765309 1597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:38.765679 1597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:38.768918 1597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "dffe5b7cef6d435bbd23294d65feea49"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "fa532bb67642a378c98322e57264d5dc"
server_key_iv: "6ada7028238173fda78dc6b8f5196e3c"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.769879 1597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "dffe5b7cef6d435bbd23294d65feea49"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "fa532bb67642a378c98322e57264d5dc"
server_key_iv: "6ada7028238173fda78dc6b8f5196e3c"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.775777 1597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:43:38.780493 1614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.782277 1597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:43:38.782518 1597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "dffe5b7cef6d435bbd23294d65feea49"
format_stamp: "Formatted at 2026-08-07 19:43:38 on dist-test-slave-btw8"
server_key: "fa532bb67642a378c98322e57264d5dc"
server_key_iv: "6ada7028238173fda78dc6b8f5196e3c"
server_key_version: "encryptionkey@0"
I20260807 19:43:38.782786 1597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:38.840678 1597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:38.845726 1597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:38.846071 1597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:38.847200 1597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:38.848892 1597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:38.849016 1597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.849188 1597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:38.849246 1597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:38.885887 1597 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:34543
I20260807 19:43:38.885937 1730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:34543 every 8 connection(s)
I20260807 19:43:38.887444 1597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:38.890519 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1597
I20260807 19:43:38.890772 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:38.891283 795 external_mini_cluster.cc:1428] Setting key d079019c5c688952e3a908cf584efff6
Aug 07 19:43:38 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:38.914850 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:38.890249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:53667 (local address 127.0.198.254:41977)
0807 19:43:38.890849 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:38.890858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:38.892032 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 19:43:38.894414 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:38.894423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:38.894509 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:38.894604 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:38.896417 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.897912 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.899988 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.900872 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.901051 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:38.905484 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:38.905495 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:38.905504 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:38.905566 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:38.910020 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:38.911159 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:38.911169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:38.911176 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:38.911429 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:38.912145 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:38.912154 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:38.912161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:38.912707 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:38.912900 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:38.913895 (+ 995us) server_negotiation.cc:299] Negotiation successful
0807 19:43:38.914240 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":276,"threads_started":1}
I20260807 19:43:38.915534 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:38.890906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:41977 (local address 127.0.198.194:53667)
0807 19:43:38.891822 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:38.891880 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:38.894117 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:38.894899 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:38.894908 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:38.895562 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:38.896191 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:38.896210 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.898182 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.898187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:38.899774 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:38.899792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:38.901197 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:38.901204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:38.901439 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:38.902303 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:38.902331 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:38.905124 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:38.910250 (+ 5126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:38.910258 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:38.910276 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:38.910967 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:38.911723 (+ 756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:38.911728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:38.911730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:38.911977 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:38.912887 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:38.912895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:38.913420 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 19:43:38.913684 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 19:43:38.914190 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":404,"threads_started":1}
I20260807 19:43:38.917351 1731 heartbeater.cc:347] Connected to a master server at 127.0.198.254:41977
I20260807 19:43:38.917681 1731 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
I20260807 19:43:38.918702 1731 heartbeater.cc:511] Master 127.0.198.254:41977 requested a full tablet report, sending...
I20260807 19:43:38.920506 1391 ts_manager.cc:194] Registered new tserver with Master: dffe5b7cef6d435bbd23294d65feea49 (127.0.198.194:34543)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
I20260807 19:43:38.921541 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:53667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:43:38.952306 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:39.317446 1738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:39.317768 1738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:39.317943 1738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:39.325618 1738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:39.325750 1738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:39.325965 1738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:43:39.330039 1590 heartbeater.cc:503] Master 127.0.198.254:41977 was elected leader, sending a full tablet report...
I20260807 19:43:39.336797 1738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:39.338129 1738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:39.339438 1738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:39.354106 1744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:39.354130 1745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:39.354060 1747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:39.354948 1738 server_base.cc:1034] running on GCE node
I20260807 19:43:39.355715 1738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:39.356794 1738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:39.358089 1738 hybrid_clock.cc:648] HybridClock initialized: now 1786131819357997 us; error 66 us; skew 500 ppm
I20260807 19:43:39.358505 1738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:39.362295 1738 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:43:39.363989 1738 webserver.cc:460] Webserver started at http://127.0.198.195:38605/ using document root <none> and password file <none>
I20260807 19:43:39.364754 1738 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:39.364866 1738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:39.365167 1738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:39.368659 1738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "3ac831b5db9c400ab4c2a76db491b0b1"
format_stamp: "Formatted at 2026-08-07 19:43:39 on dist-test-slave-btw8"
server_key: "e102548ec5753cbe64b2afd3f34b0d32"
server_key_iv: "fddc9dc10cf0dfe9030657c24819c13f"
server_key_version: "encryptionkey@0"
I20260807 19:43:39.369652 1738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "3ac831b5db9c400ab4c2a76db491b0b1"
format_stamp: "Formatted at 2026-08-07 19:43:39 on dist-test-slave-btw8"
server_key: "e102548ec5753cbe64b2afd3f34b0d32"
server_key_iv: "fddc9dc10cf0dfe9030657c24819c13f"
server_key_version: "encryptionkey@0"
I20260807 19:43:39.375988 1738 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:43:39.380717 1755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:39.382375 1738 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:43:39.382596 1738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "3ac831b5db9c400ab4c2a76db491b0b1"
format_stamp: "Formatted at 2026-08-07 19:43:39 on dist-test-slave-btw8"
server_key: "e102548ec5753cbe64b2afd3f34b0d32"
server_key_iv: "fddc9dc10cf0dfe9030657c24819c13f"
server_key_version: "encryptionkey@0"
I20260807 19:43:39.382844 1738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:39.415809 1738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:39.421401 1738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:39.421782 1738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:39.422962 1738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:39.424650 1738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:39.424738 1738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:39.424888 1738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:39.424942 1738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:39.468946 1738 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:41507
I20260807 19:43:39.469012 1880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:41507 every 8 connection(s)
I20260807 19:43:39.470474 1738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:39.478329 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1738
I20260807 19:43:39.478632 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:39.479259 795 external_mini_cluster.cc:1428] Setting key cb287ea4ef5f16944e9885f9d9612718
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:39.497895 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.473821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:47843 (local address 127.0.198.254:41977)
0807 19:43:39.474401 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.474408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.475944 (+ 1536us) server_negotiation.cc:407] Connection header received
0807 19:43:39.478408 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.478413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.478503 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.478656 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.481067 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.482238 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.484494 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.485032 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.485146 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.489464 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.489471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.489475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.489521 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.492942 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.494117 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.494124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.494128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.494336 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.495119 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.495126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.495129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.495648 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.495840 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.497219 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.497569 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":272,"threads_started":1}
I20260807 19:43:39.498817 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.474748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:41977 (local address 127.0.198.195:47843)
0807 19:43:39.475755 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.475809 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.477911 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.478999 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.479013 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.480011 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:39.480799 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.480829 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.482453 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.482459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.484326 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.484341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.485293 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.485300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.485524 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.486420 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:39.486459 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:39.489118 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:39.493182 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:39.493190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:39.493208 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:39.493845 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:39.494586 (+ 741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:39.494591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:39.494595 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:39.494956 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:39.495805 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:39.495812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:39.496423 (+ 611us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.497023 (+ 600us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.497606 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":340,"threads_started":1}
I20260807 19:43:39.500502 1881 heartbeater.cc:347] Connected to a master server at 127.0.198.254:41977
I20260807 19:43:39.500917 1881 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:39.501751 1881 heartbeater.cc:511] Master 127.0.198.254:41977 requested a full tablet report, sending...
I20260807 19:43:39.503623 1391 ts_manager.cc:194] Registered new tserver with Master: 3ac831b5db9c400ab4c2a76db491b0b1 (127.0.198.195:41507)
I20260807 19:43:39.504616 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:47843
I20260807 19:43:39.510957 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:43:39.536196 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.520010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.0.198.254:41977)
0807 19:43:39.520272 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.520279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.520420 (+ 141us) server_negotiation.cc:407] Connection header received
0807 19:43:39.520727 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.520732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.520803 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.520929 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.522601 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.523437 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.525287 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.525897 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.526052 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.528810 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.528815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.528819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.528862 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.532669 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.533607 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.533619 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.533622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.533790 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.534267 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.534275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.534279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.534703 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.535007 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.535457 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.535848 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 19:43:39.545876 1391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:39.555182 1391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:39.593837 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.569956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:41507 (local address 127.0.0.1:43566)
0807 19:43:39.583471 (+ 13515us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.583497 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.583729 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.584860 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.584867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.584919 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:39.585415 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.585431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.587441 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.587449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.590338 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.590354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.592553 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.592559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.592812 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.592925 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.593080 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.593295 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13357,"mutex_wait_us":28,"spinlock_wait_cycles":11904,"thread_start_us":13057,"threads_started":1}
I20260807 19:43:39.593827 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.571077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.0.198.195:41507)
0807 19:43:39.582142 (+ 11065us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.582158 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.583900 (+ 1742us) server_negotiation.cc:407] Connection header received
0807 19:43:39.584011 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.584019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.584284 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.584521 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:39.585672 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.587191 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.590541 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.592322 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.592485 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.592789 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.593158 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.593366 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10758,"thread_start_us":10520,"threads_started":1}
I20260807 19:43:39.594399 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.570221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.0.198.193:34691)
0807 19:43:39.582711 (+ 12490us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.582725 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.583907 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 19:43:39.584058 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.584069 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.584316 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.584582 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:39.586252 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.588553 (+ 2301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.590791 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.592445 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.592580 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.592794 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.593847 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.594049 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12235,"thread_start_us":12027,"threads_started":1}
I20260807 19:43:39.594645 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.569897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:34691 (local address 127.0.0.1:48784)
0807 19:43:39.583312 (+ 13415us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.583395 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.583656 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.584860 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.584867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.584912 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:39.586016 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.586029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.588710 (+ 2681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.588719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.590554 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.590572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.592858 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.592864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.593116 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.593145 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.593224 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.593333 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13092,"mutex_wait_us":543,"spinlock_wait_cycles":896,"thread_start_us":12860,"threads_started":1}
I20260807 19:43:39.596848 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.585114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:34543 (local address 127.0.0.1:48860)
0807 19:43:39.585841 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.585866 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.586041 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.587152 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.587157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.587185 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:39.588540 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.588584 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.590556 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.590560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.593028 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.593071 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.595904 (+ 2833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.595911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.596108 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.596144 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.596245 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.596399 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":122,"thread_start_us":415,"threads_started":1}
I20260807 19:43:39.597620 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.584215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.0.198.194:34543)
0807 19:43:39.585396 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.585405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.585985 (+ 580us) server_negotiation.cc:407] Connection header received
0807 19:43:39.586223 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.586231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.586507 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.586732 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:39.588802 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.590356 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.593278 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.595272 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.595479 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.595718 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.597199 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.597353 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":884,"threads_started":1}
I20260807 19:43:39.598603 1806 tablet_service.cc:1467] Processing CreateTablet for tablet aea809317e854091b35631b5123837ff (DEFAULT_TABLE table=test-table [id=4c2db28ebc9e482e97cf5158d3cda6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:39.598595 1519 tablet_service.cc:1467] Processing CreateTablet for tablet aea809317e854091b35631b5123837ff (DEFAULT_TABLE table=test-table [id=4c2db28ebc9e482e97cf5158d3cda6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:39.600196 1519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea809317e854091b35631b5123837ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:39.600210 1806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea809317e854091b35631b5123837ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:39.601060 1664 tablet_service.cc:1467] Processing CreateTablet for tablet aea809317e854091b35631b5123837ff (DEFAULT_TABLE table=test-table [id=4c2db28ebc9e482e97cf5158d3cda6cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:39.602639 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea809317e854091b35631b5123837ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:39.622673 1903 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Bootstrap starting.
I20260807 19:43:39.624994 1902 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Bootstrap starting.
I20260807 19:43:39.628985 1902 tablet_bootstrap.cc:654] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:39.630723 1902 log.cc:824] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:39.632313 1903 tablet_bootstrap.cc:654] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:39.633808 1904 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Bootstrap starting.
I20260807 19:43:39.635013 1903 log.cc:824] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:39.640741 1904 tablet_bootstrap.cc:654] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:39.641076 1902 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: No bootstrap required, opened a new log
I20260807 19:43:39.641453 1902 ts_tablet_manager.cc:1397] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.000s
I20260807 19:43:39.642266 1904 log.cc:824] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:39.646565 1903 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: No bootstrap required, opened a new log
I20260807 19:43:39.646674 1904 tablet_bootstrap.cc:492] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: No bootstrap required, opened a new log
I20260807 19:43:39.647058 1903 ts_tablet_manager.cc:1397] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.004s
I20260807 19:43:39.647039 1904 ts_tablet_manager.cc:1397] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260807 19:43:39.650919 1902 raft_consensus.cc:348] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.651602 1902 raft_consensus.cc:374] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:39.651724 1902 raft_consensus.cc:729] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fe5c0aaf1d141be9e082fc76d562ec6, State: Initialized, Role: FOLLOWER
I20260807 19:43:39.652446 1902 consensus_queue.cc:260] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.654492 1902 ts_tablet_manager.cc:1428] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260807 19:43:39.660347 1903 raft_consensus.cc:348] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.660351 1904 raft_consensus.cc:348] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.660970 1903 raft_consensus.cc:374] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:39.660970 1904 raft_consensus.cc:374] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:39.661101 1904 raft_consensus.cc:729] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffe5b7cef6d435bbd23294d65feea49, State: Initialized, Role: FOLLOWER
I20260807 19:43:39.661111 1903 raft_consensus.cc:729] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac831b5db9c400ab4c2a76db491b0b1, State: Initialized, Role: FOLLOWER
I20260807 19:43:39.661911 1904 consensus_queue.cc:260] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.661911 1903 consensus_queue.cc:260] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.663339 1881 heartbeater.cc:503] Master 127.0.198.254:41977 was elected leader, sending a full tablet report...
I20260807 19:43:39.663939 1731 heartbeater.cc:503] Master 127.0.198.254:41977 was elected leader, sending a full tablet report...
I20260807 19:43:39.664400 1903 ts_tablet_manager.cc:1428] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 19:43:39.664589 1904 ts_tablet_manager.cc:1428] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260807 19:43:39.685452 1910 raft_consensus.cc:482] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:39.685853 1910 raft_consensus.cc:504] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.688207 1910 leader_election.cc:288] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fe5c0aaf1d141be9e082fc76d562ec6 (127.0.198.193:34691), 3ac831b5db9c400ab4c2a76db491b0b1 (127.0.198.195:41507)
I20260807 19:43:39.696430 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.688841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:41507 (local address 127.0.198.194:48061)
0807 19:43:39.689819 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.689835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.689975 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.690348 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.690353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.690378 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:39.690949 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.690961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.692196 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.692202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.693925 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.693938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.695542 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.695549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.695729 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.695766 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.695882 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.696047 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":36,"spinlock_wait_cycles":25856,"thread_start_us":520,"threads_started":1}
I20260807 19:43:39.696568 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.688791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:48061 (local address 127.0.198.195:41507)
0807 19:43:39.689070 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.689078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.689941 (+ 863us) server_negotiation.cc:407] Connection header received
0807 19:43:39.690129 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.690134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.690223 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.690316 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:39.691124 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.692017 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.694163 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.695252 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.695480 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.695545 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.695972 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.696279 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 19:43:39.697069 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.688845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:47507 (local address 127.0.198.193:34691)
0807 19:43:39.689080 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.689088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.690111 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 19:43:39.690297 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.690304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.690386 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.690492 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:39.691326 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.692402 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.694715 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.695789 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.695943 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.696011 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.696695 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.696859 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:43:39.697784 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.688841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:34691 (local address 127.0.198.194:47507)
0807 19:43:39.689948 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:39.689968 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:39.690137 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:39.690533 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:39.690536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:39.690578 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:39.691104 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.691131 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.692581 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.692588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.694458 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:39.694475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.696187 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.696203 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:39.696444 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.696482 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:43:39.697357 (+ 875us) client_negotiation.cc:240] Negotiation successful
0807 19:43:39.697478 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":189,"thread_start_us":493,"threads_started":1}
I20260807 19:43:39.697926 1829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aea809317e854091b35631b5123837ff" candidate_uuid: "dffe5b7cef6d435bbd23294d65feea49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" is_pre_election: true
I20260807 19:43:39.698582 1829 raft_consensus.cc:2454] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dffe5b7cef6d435bbd23294d65feea49 in term 0.
I20260807 19:43:39.699081 1541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aea809317e854091b35631b5123837ff" candidate_uuid: "dffe5b7cef6d435bbd23294d65feea49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" is_pre_election: true
I20260807 19:43:39.699393 1618 leader_election.cc:302] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac831b5db9c400ab4c2a76db491b0b1, dffe5b7cef6d435bbd23294d65feea49; no voters:
I20260807 19:43:39.699781 1541 raft_consensus.cc:2454] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dffe5b7cef6d435bbd23294d65feea49 in term 0.
I20260807 19:43:39.699880 1910 raft_consensus.cc:2781] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:39.700031 1910 raft_consensus.cc:482] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:39.700245 1910 raft_consensus.cc:3037] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:39.702759 1910 raft_consensus.cc:504] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.703590 1910 leader_election.cc:288] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [CANDIDATE]: Term 1 election: Requested vote from peers 4fe5c0aaf1d141be9e082fc76d562ec6 (127.0.198.193:34691), 3ac831b5db9c400ab4c2a76db491b0b1 (127.0.198.195:41507)
I20260807 19:43:39.704087 1541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aea809317e854091b35631b5123837ff" candidate_uuid: "dffe5b7cef6d435bbd23294d65feea49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6"
I20260807 19:43:39.704161 1829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aea809317e854091b35631b5123837ff" candidate_uuid: "dffe5b7cef6d435bbd23294d65feea49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac831b5db9c400ab4c2a76db491b0b1"
I20260807 19:43:39.704350 1541 raft_consensus.cc:3037] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:39.704434 1829 raft_consensus.cc:3037] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:39.707067 1829 raft_consensus.cc:2454] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dffe5b7cef6d435bbd23294d65feea49 in term 1.
I20260807 19:43:39.707084 1541 raft_consensus.cc:2454] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dffe5b7cef6d435bbd23294d65feea49 in term 1.
I20260807 19:43:39.707744 1617 leader_election.cc:302] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fe5c0aaf1d141be9e082fc76d562ec6, dffe5b7cef6d435bbd23294d65feea49; no voters:
I20260807 19:43:39.708237 1910 raft_consensus.cc:2781] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:39.708818 1910 raft_consensus.cc:686] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [term 1 LEADER]: Becoming Leader. State: Replica: dffe5b7cef6d435bbd23294d65feea49, State: Running, Role: LEADER
I20260807 19:43:39.709389 1910 consensus_queue.cc:237] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } }
I20260807 19:43:39.715310 1389 catalog_manager.cc:5495] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49 reported cstate change: term changed from 0 to 1, leader changed from <none> to dffe5b7cef6d435bbd23294d65feea49 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "dffe5b7cef6d435bbd23294d65feea49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fe5c0aaf1d141be9e082fc76d562ec6" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ac831b5db9c400ab4c2a76db491b0b1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dffe5b7cef6d435bbd23294d65feea49" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34543 } health_report { overall_health: HEALTHY } } }
W20260807 19:43:39.722622 1882 tablet.cc:2369] T aea809317e854091b35631b5123837ff P 3ac831b5db9c400ab4c2a76db491b0b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:39.765911 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.746917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.0.198.193:34691)
0807 19:43:39.747149 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.747154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.747834 (+ 680us) server_negotiation.cc:407] Connection header received
0807 19:43:39.748088 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.748093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.748165 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.748284 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.750062 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.751112 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.752765 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.753283 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.753452 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.758526 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.758547 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.758566 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.758623 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.762269 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.763298 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.763307 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.763310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.763591 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.764170 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.764173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.764176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.764683 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.764996 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.765315 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.765546 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 19:43:39.792762 1592 tablet.cc:2369] T aea809317e854091b35631b5123837ff P 4fe5c0aaf1d141be9e082fc76d562ec6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:39.826660 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.809224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.0.198.193:34691)
0807 19:43:39.809543 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.809551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.809847 (+ 296us) server_negotiation.cc:407] Connection header received
0807 19:43:39.810013 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.810017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.810092 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.810199 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.812014 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.812828 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.814448 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.815009 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.815119 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.820231 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.820237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.820240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.820286 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.823000 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.824224 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.824229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.824231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.824384 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.824953 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.824958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.824960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.825293 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.825513 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.825997 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.826214 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:43:39 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
W20260807 19:43:39.891218 1732 tablet.cc:2369] T aea809317e854091b35631b5123837ff P dffe5b7cef6d435bbd23294d65feea49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:39.891502 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.872425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.0.198.193:34691)
0807 19:43:39.872714 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.872722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.872966 (+ 244us) server_negotiation.cc:407] Connection header received
0807 19:43:39.873204 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.873209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.873283 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.873390 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.875103 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.876248 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.878105 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.878669 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.878785 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.883853 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.883860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.883864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.883923 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.887526 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.888680 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.888687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.888690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.888849 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.889328 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.889340 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.889343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.889968 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.890323 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.890716 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.891029 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 19:43:39.893518 1497 tablet_service.cc:2908] Rejecting scan request for tablet aea809317e854091b35631b5123837ff: Uninitialized: clean time has not yet been initialized
I20260807 19:43:39.897231 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1453
I20260807 19:43:39.914279 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:39.898296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.0.198.193:34691)
0807 19:43:39.898499 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:39.898504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:39.898988 (+ 484us) server_negotiation.cc:407] Connection header received
0807 19:43:39.899222 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:39.899229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:39.899329 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:39.899511 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:39.900998 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.901788 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.903194 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:39.903771 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:39.903933 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:39.907188 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:39.907200 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:39.907205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:39.907254 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:39.911178 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.912120 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.912125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.912127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.912301 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:39.912818 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:39.912821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:39.912823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:39.913139 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:39.913395 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:39.913799 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:43:39.913981 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:43:39.915112 1584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:40.018167 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1453
I20260807 19:43:40.040613 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1597
Aug 07 19:43:40 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:43:40.060314 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:40.041614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.0.198.194:34543)
0807 19:43:40.041843 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:40.041848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:40.041867 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:40.042087 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:40.042092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:40.042176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:40.042302 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:40.043768 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:40.044625 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:40.046300 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:40.046797 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:40.046916 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:40.052498 (+ 5582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:40.052518 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:40.052536 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:40.052597 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:40.056651 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:40.057725 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:40.057735 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:40.057737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:40.057937 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:40.058455 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:40.058459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:40.058462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:40.058934 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:40.059271 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:40.059719 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 19:43:40.059986 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:43:40.061185 1725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:40.157666 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:40.157140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:34691 (local address 127.0.198.194:49231)
0807 19:43:40.157331 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:40.157482 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:34691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:43:40.157635 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1597
W20260807 19:43:40.177984 1759 connection.cc:535] server connection from 127.0.198.194:48061 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:43:40.178602 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1738
Aug 07 19:43:40 dist-test-slave-btw8 krb5kdc[1346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:43:40.197177 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:40.179681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.0.198.195:41507)
0807 19:43:40.179911 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:40.179916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:40.179936 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:40.180117 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:40.180121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:40.180190 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:40.180283 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:40.181726 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:40.182569 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:40.184195 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:40.184613 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:40.184725 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:40.190054 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:40.190074 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:40.190093 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:40.190147 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:40.193672 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:40.194590 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:40.194599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:40.194602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:40.194786 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:40.195289 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:40.195293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:40.195296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:40.195791 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:40.196140 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:40.196513 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 19:43:40.196784 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 19:43:40.198105 1875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:40.292176 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1738
I20260807 19:43:40.312108 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1359
I20260807 19:43:40.313476 1422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:40.427940 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1359
2026-08-07T19:43:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4734 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:40 dist-test-slave-btw8 krb5kdc[1935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:40 dist-test-slave-btw8 krb5kdc[1935](info): set up 2 sockets
Aug 07 19:43:40 dist-test-slave-btw8 krb5kdc[1935](info): commencing operation
krb5kdc: starting...
W20260807 19:43:41.600132 795 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:43:41 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:43:41.860546 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37481
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:42.225562 1949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:42.225929 1949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:42.226123 1949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:42.234018 1949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:42.234170 1949 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:43:42.234261 1949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:42.234378 1949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:42.234465 1949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:42.234571 1949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:42.245349 1949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:37481
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:42.246862 1949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:42.248402 1949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:42.262284 1955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:42.262327 1958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:42.262236 1956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:42.263839 1949 server_base.cc:1034] running on GCE node
I20260807 19:43:42.264798 1949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:42.266426 1949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:42.267714 1949 hybrid_clock.cc:648] HybridClock initialized: now 1786131822267615 us; error 53 us; skew 500 ppm
I20260807 19:43:42.268144 1949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:42 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:42.271727 1949 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:43:42.273361 1949 webserver.cc:460] Webserver started at http://127.0.198.254:39823/ using document root <none> and password file <none>
I20260807 19:43:42.274138 1949 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:42.274235 1949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:42.274591 1949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:42.277750 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "4af3385d5d1f4b5fa9c72072de2598e2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "04e783c9661c74bd4e3ec37d4d6790dc"
server_key_iv: "9cc966193e02e3c9610f01c6a0c26c89"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.278644 1949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "4af3385d5d1f4b5fa9c72072de2598e2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "04e783c9661c74bd4e3ec37d4d6790dc"
server_key_iv: "9cc966193e02e3c9610f01c6a0c26c89"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.284179 1949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:43:42.287920 1965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:42.289395 1949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:43:42.289598 1949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "4af3385d5d1f4b5fa9c72072de2598e2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "04e783c9661c74bd4e3ec37d4d6790dc"
server_key_iv: "9cc966193e02e3c9610f01c6a0c26c89"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.289850 1949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:42.306552 1949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:42.311722 1949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:42.312053 1949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:42.338555 1949 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:37481
I20260807 19:43:42.338593 2034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:37481 every 8 connection(s)
I20260807 19:43:42.340224 1949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:42.340991 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 1949
I20260807 19:43:42.341249 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:43:42.342156 795 external_mini_cluster.cc:1428] Setting key 2ecda9e34c365e976414e957674dbaf6
I20260807 19:43:42.345393 2035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:43:42 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:42.357646 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Bootstrap starting.
I20260807 19:43:42.361330 2035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:42.362494 2035 log.cc:824] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:42.365516 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:42.344400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.0.198.254:37481)
0807 19:43:42.345267 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:42.345279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:42.345321 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:43:42.347088 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:42.347111 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:42.347583 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:42.348016 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:42.349573 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:42.350992 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:42.352581 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:42.353275 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:42.353485 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:42.357876 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:42.357894 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:42.357911 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:42.357958 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:42.361259 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:42.362043 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:42.362050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:42.362057 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:42.362243 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:42.362665 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:42.362671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:42.362676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:42.363236 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:42.363613 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:42.364078 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 19:43:42.364521 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":421,"threads_started":1}
I20260807 19:43:42.365990 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: No bootstrap required, opened a new log
I20260807 19:43:42.375784 2035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } }
I20260807 19:43:42.376282 2035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:42.376405 2035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af3385d5d1f4b5fa9c72072de2598e2, State: Initialized, Role: FOLLOWER
I20260807 19:43:42.377142 2035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } }
I20260807 19:43:42.377497 2035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:42.377691 2035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:42.377926 2035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:42.380576 2035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } }
I20260807 19:43:42.381117 2035 leader_election.cc:302] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4af3385d5d1f4b5fa9c72072de2598e2; no voters:
I20260807 19:43:42.381812 2035 leader_election.cc:288] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:42.382025 2040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:42.383481 2040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [term 1 LEADER]: Becoming Leader. State: Replica: 4af3385d5d1f4b5fa9c72072de2598e2, State: Running, Role: LEADER
I20260807 19:43:42.384426 2035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:42.384339 2040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } }
I20260807 19:43:42.387567 2041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } } }
I20260807 19:43:42.388013 2041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:42.388929 2049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:42.388787 2042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4af3385d5d1f4b5fa9c72072de2598e2. Latest consensus state: current_term: 1 leader_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af3385d5d1f4b5fa9c72072de2598e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37481 } } }
I20260807 19:43:42.389178 2042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:42.394510 2049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:42.403609 2049 catalog_manager.cc:1347] Generated new cluster ID: 21f8e20d90fb42c7911e199d6b6536de
I20260807 19:43:42.403761 2049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:42.422003 2049 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:42.424100 2049 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:42.434221 2049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Generated new TSK 0
I20260807 19:43:42.435235 2049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:43:42.508297 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37481
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:42.883667 2062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:42.883993 2062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:42.884150 2062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:42.891930 2062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:42.892050 2062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:42.892237 2062 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:42.892371 2062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:43:42.902787 2062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:42.904341 2062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:42.905743 2062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:42.920183 2067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:42.920300 2068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:42.920215 2070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:42.921321 2062 server_base.cc:1034] running on GCE node
I20260807 19:43:42.921923 2062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:42.922889 2062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:42.924104 2062 hybrid_clock.cc:648] HybridClock initialized: now 1786131822924038 us; error 51 us; skew 500 ppm
I20260807 19:43:42.924420 2062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:42 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:42.927801 2062 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:43:42.929440 2062 webserver.cc:460] Webserver started at http://127.0.198.193:45385/ using document root <none> and password file <none>
I20260807 19:43:42.930153 2062 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:42.930290 2062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:42.930740 2062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:42.933791 2062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "31b2e1919f27432d8518403715a061b2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "7cb270239b7efc5abbc5ce8fd5e6549e"
server_key_iv: "d46cf6d69304aa68bee8e97b3612b34b"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.934791 2062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "31b2e1919f27432d8518403715a061b2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "7cb270239b7efc5abbc5ce8fd5e6549e"
server_key_iv: "d46cf6d69304aa68bee8e97b3612b34b"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.940212 2062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:43:42.943838 2077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:42.945317 2062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:43:42.945503 2062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "31b2e1919f27432d8518403715a061b2"
format_stamp: "Formatted at 2026-08-07 19:43:42 on dist-test-slave-btw8"
server_key: "7cb270239b7efc5abbc5ce8fd5e6549e"
server_key_iv: "d46cf6d69304aa68bee8e97b3612b34b"
server_key_version: "encryptionkey@0"
I20260807 19:43:42.945760 2062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:42.982239 2062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:42.987231 2062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:42.987572 2062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:42.990785 2062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:42.990898 2062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:42.991042 2062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:42.991137 2062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:43 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:43.020280 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:42.991941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:54203 (local address 127.0.198.254:37481)
0807 19:43:42.992481 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:42.992490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:42.994046 (+ 1556us) server_negotiation.cc:407] Connection header received
0807 19:43:42.997276 (+ 3230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:42.997285 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:42.997409 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:42.997561 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:42.999302 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.000616 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.003881 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.004590 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.004765 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.010619 (+ 5854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:43.010630 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:43.010638 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:43.010698 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:43.015050 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.016555 (+ 1505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.016565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.016572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.016779 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.017725 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.017731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.017734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.018192 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:43.018391 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:43.019323 (+ 932us) server_negotiation.cc:299] Negotiation successful
0807 19:43:43.019686 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
I20260807 19:43:43.021016 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:42.992720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.193:54203)
0807 19:43:42.993832 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:42.993887 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:42.996835 (+ 2948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:42.997861 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:42.997869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:42.998447 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:42.999082 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:42.999102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.000873 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.000882 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.003687 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.003713 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.005145 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.005152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.005591 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.006519 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:43.006561 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:43.010294 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:43.015290 (+ 4996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.015304 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.015328 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.016158 (+ 830us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.016932 (+ 774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.016939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.016944 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.017530 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.018375 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:43.018391 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:43.018875 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 19:43:43.019128 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 19:43:43.019686 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":484,"threads_started":1}
I20260807 19:43:43.047067 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.027866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:52783 (local address 127.0.198.254:37481)
0807 19:43:43.028157 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:43.028165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:43.029676 (+ 1511us) server_negotiation.cc:407] Connection header received
0807 19:43:43.029987 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:43.029995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:43.030114 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:43.030235 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:43.032006 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.032711 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.035341 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.036047 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.036201 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.039370 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:43.039379 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:43.039386 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:43.039440 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:43.043381 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.044123 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.044131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.044137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.044312 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.044796 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.044804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.044809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.045268 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:43.045455 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:43.046148 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 19:43:43.046564 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260807 19:43:43.047176 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.028337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.193:52783)
0807 19:43:43.029528 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:43.029552 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:43.029794 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:43.030297 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:43.030305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:43.031181 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:43.031814 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.031832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.032861 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.032870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.035162 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.035187 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.036524 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.036534 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.036745 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.037578 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:43.037607 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:43.039104 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:43.043533 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.043540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.043547 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.043995 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.044436 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.044443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.044448 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.044650 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.045437 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:43.045448 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:43.045788 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 19:43:43.046004 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 19:43:43.046485 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":804,"threads_started":1}
I20260807 19:43:43.050343 2062 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:42809
I20260807 19:43:43.050464 2204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:42809 every 8 connection(s)
I20260807 19:43:43.052096 2062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:43.054550 2205 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37481
I20260807 19:43:43.054912 2205 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:43.055688 2205 heartbeater.cc:511] Master 127.0.198.254:37481 requested a full tablet report, sending...
I20260807 19:43:43.056924 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2062
I20260807 19:43:43.057170 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:43.057823 795 external_mini_cluster.cc:1428] Setting key 56985a09b154d67091efe4a5ffcc7eb4
I20260807 19:43:43.058255 1985 ts_manager.cc:194] Registered new tserver with Master: 31b2e1919f27432d8518403715a061b2 (127.0.198.193:42809)
I20260807 19:43:43.060683 1985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:54203
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:43:43.112157 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37481
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:43.473659 2209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:43.473989 2209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:43.474164 2209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:43.482466 2209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:43.482632 2209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:43.482853 2209 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:43.483031 2209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:43:43.495309 2209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:43.496855 2209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:43.498430 2209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:43.515106 2215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:43.515780 2214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:43.516093 2217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:43.517732 2209 server_base.cc:1034] running on GCE node
I20260807 19:43:43.518699 2209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:43.520150 2209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:43.521688 2209 hybrid_clock.cc:648] HybridClock initialized: now 1786131823521558 us; error 75 us; skew 500 ppm
I20260807 19:43:43.522231 2209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:43 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:43.531018 2209 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:43:43.533229 2209 webserver.cc:460] Webserver started at http://127.0.198.194:36805/ using document root <none> and password file <none>
I20260807 19:43:43.534089 2209 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:43.534250 2209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:43.534641 2209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:43.538165 2209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "4c1fe2e357e34cc2a9beeab70f35e611"
format_stamp: "Formatted at 2026-08-07 19:43:43 on dist-test-slave-btw8"
server_key: "b5a4bff8bc6b389c6fcae60d9ecf770f"
server_key_iv: "11cb143f4477bd44146635fae51b22a9"
server_key_version: "encryptionkey@0"
I20260807 19:43:43.539141 2209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "4c1fe2e357e34cc2a9beeab70f35e611"
format_stamp: "Formatted at 2026-08-07 19:43:43 on dist-test-slave-btw8"
server_key: "b5a4bff8bc6b389c6fcae60d9ecf770f"
server_key_iv: "11cb143f4477bd44146635fae51b22a9"
server_key_version: "encryptionkey@0"
I20260807 19:43:43.545373 2209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:43:43.550318 2224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:43.552193 2209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:43:43.552423 2209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "4c1fe2e357e34cc2a9beeab70f35e611"
format_stamp: "Formatted at 2026-08-07 19:43:43 on dist-test-slave-btw8"
server_key: "b5a4bff8bc6b389c6fcae60d9ecf770f"
server_key_iv: "11cb143f4477bd44146635fae51b22a9"
server_key_version: "encryptionkey@0"
I20260807 19:43:43.552699 2209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:43.568390 2209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:43.573299 2209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:43.573618 2209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:43.576678 2209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:43.576798 2209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:43.576936 2209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:43.577018 2209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:43 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:43.604647 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.577388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:53985 (local address 127.0.198.254:37481)
0807 19:43:43.578079 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:43.578086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:43.579310 (+ 1224us) server_negotiation.cc:407] Connection header received
0807 19:43:43.581500 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:43.581506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:43.581610 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:43.581741 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:43.584204 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.585654 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.588537 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.589456 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.589648 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.595278 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:43.595285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:43.595288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:43.595329 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:43.599077 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.600548 (+ 1471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.600557 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.600563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.600756 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.601579 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.601594 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.601596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.602084 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:43.602305 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:43.603672 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0807 19:43:43.604036 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":466,"threads_started":1}
I20260807 19:43:43.605582 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.578164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.194:53985)
0807 19:43:43.579109 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:43.579155 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:43.581162 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:43.582086 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:43.582102 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:43.583035 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:43.583948 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.583984 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.585907 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.585912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.588398 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.588412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.589732 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.589738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.589948 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.590872 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:43.590897 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:43.594927 (+ 4030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:43.599310 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.599327 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.599381 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.600313 (+ 932us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.600919 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.600930 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.600937 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.601389 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.602394 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:43.602402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:43.603094 (+ 692us) client_negotiation.cc:769] Sending connection context
0807 19:43:43.603512 (+ 418us) client_negotiation.cc:240] Negotiation successful
0807 19:43:43.604143 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":381,"threads_started":1}
I20260807 19:43:43.632763 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.614531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.194:50423)
0807 19:43:43.615838 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:43.615857 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:43.616082 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:43.616691 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:43.616710 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:43.617786 (+ 1076us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:43.618395 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.618421 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.619739 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.619745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.621598 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:43.621627 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.622799 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.622807 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:43.622977 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.623960 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:43.623982 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:43.624926 (+ 944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:43.628999 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.629008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.629013 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.629651 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.630214 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:43.630221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:43.630225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:43.630405 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:43.631226 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:43.631232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:43.631618 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 19:43:43.631906 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 19:43:43.632294 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":1029,"threads_started":1}
I20260807 19:43:43.632778 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:43.614369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:50423 (local address 127.0.198.254:37481)
0807 19:43:43.614610 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:43.614617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:43.615978 (+ 1361us) server_negotiation.cc:407] Connection header received
0807 19:43:43.616266 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:43.616277 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:43.616399 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:43.616530 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:43.618677 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.619560 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.621882 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:43.622557 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:43.622715 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:43.625189 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:43.625195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:43.625199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:43.625237 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:43.628754 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.629844 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.629854 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.629859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.630052 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:43.630587 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:43.630595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:43.630599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:43.631051 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:43.631310 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:43.632045 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 19:43:43.632382 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:43:43.642408 2209 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:43231
I20260807 19:43:43.642424 2350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:43231 every 8 connection(s)
I20260807 19:43:43.643890 2209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:43.646584 2351 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37481
I20260807 19:43:43.646883 2351 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:43.647744 2351 heartbeater.cc:511] Master 127.0.198.254:37481 requested a full tablet report, sending...
I20260807 19:43:43.648192 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2209
I20260807 19:43:43.648415 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:43.648909 795 external_mini_cluster.cc:1428] Setting key 9f8e95d2964112b645e0cc27b4e55d25
I20260807 19:43:43.649626 1985 ts_manager.cc:194] Registered new tserver with Master: 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231)
I20260807 19:43:43.650754 1985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:53985
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:43:43.706911 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37481
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:44.058017 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:44.058323 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:44.058481 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:43:44.063905 2205 heartbeater.cc:503] Master 127.0.198.254:37481 was elected leader, sending a full tablet report...
W20260807 19:43:44.067112 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:44.067241 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:44.067406 2355 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:44.067530 2355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:43:44.078083 2355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:44.079535 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:44.080961 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:44.096207 2361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:44.096236 2360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:44.096903 2363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:44.097034 2355 server_base.cc:1034] running on GCE node
I20260807 19:43:44.097700 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:44.098717 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:44.099958 2355 hybrid_clock.cc:648] HybridClock initialized: now 1786131824099890 us; error 33 us; skew 500 ppm
I20260807 19:43:44.100276 2355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:44 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:44.103729 2355 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:43:44.105206 2355 webserver.cc:460] Webserver started at http://127.0.198.195:44149/ using document root <none> and password file <none>
I20260807 19:43:44.105913 2355 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:44.106001 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:44.106272 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:44.109551 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "4d47646cfca741daa8db71b287cbea76"
format_stamp: "Formatted at 2026-08-07 19:43:44 on dist-test-slave-btw8"
server_key: "df8d7ac4d70be2c567958fc345166356"
server_key_iv: "9fc56cf5f4bbad07002c146487e8e001"
server_key_version: "encryptionkey@0"
I20260807 19:43:44.110401 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "4d47646cfca741daa8db71b287cbea76"
format_stamp: "Formatted at 2026-08-07 19:43:44 on dist-test-slave-btw8"
server_key: "df8d7ac4d70be2c567958fc345166356"
server_key_iv: "9fc56cf5f4bbad07002c146487e8e001"
server_key_version: "encryptionkey@0"
I20260807 19:43:44.116436 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:43:44.120975 2370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:44.122673 2355 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 19:43:44.122903 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "4d47646cfca741daa8db71b287cbea76"
format_stamp: "Formatted at 2026-08-07 19:43:44 on dist-test-slave-btw8"
server_key: "df8d7ac4d70be2c567958fc345166356"
server_key_iv: "9fc56cf5f4bbad07002c146487e8e001"
server_key_version: "encryptionkey@0"
I20260807 19:43:44.123246 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:44.142899 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:44.147778 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:44.148080 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:44.151145 2355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:44.151288 2355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:44.151479 2355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:44.151561 2355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:44 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:44.179463 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.151862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:46303 (local address 127.0.198.254:37481)
0807 19:43:44.154950 (+ 3088us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.154955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.154975 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:44.157457 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.157467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.157591 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.157747 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:44.160797 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.161906 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.164591 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.165331 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.165526 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.171338 (+ 5812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:44.171345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:44.171399 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:44.171455 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:44.175393 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.176262 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.176272 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.176278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.176503 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.177070 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.177078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.177084 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.177583 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:44.177769 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.178505 (+ 736us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.178826 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2972,"thread_start_us":2863,"threads_started":1}
I20260807 19:43:44.180181 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.152763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.195:46303)
0807 19:43:44.153941 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.154000 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.157020 (+ 3020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.158134 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.158146 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.159290 (+ 1144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:44.160460 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.160494 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.162162 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.162172 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.164358 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.164382 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.165681 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.165692 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.166147 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.167137 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:44.167179 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:44.171007 (+ 3828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:44.175582 (+ 4575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:44.175620 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:44.175636 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:44.176101 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:44.176681 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:44.176685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:44.176687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:44.176912 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:44.177717 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:44.177723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:44.178076 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.178324 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.178750 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":508,"threads_started":1}
I20260807 19:43:44.208786 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.193317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:34519 (local address 127.0.198.254:37481)
0807 19:43:44.193578 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.193587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.194161 (+ 574us) server_negotiation.cc:407] Connection header received
0807 19:43:44.194415 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.194421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.194507 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.194609 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:44.195690 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.196424 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.197765 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.198230 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.198378 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.201032 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:44.201038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:44.201041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:44.201075 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:44.205504 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.206476 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.206485 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.206490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.206687 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.207142 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.207145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.207147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.207490 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:44.207619 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.208169 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.208509 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:43:44.208923 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.193334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.198.195:34519)
0807 19:43:44.194016 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.194057 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.194268 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.194629 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.194634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.195161 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:44.195532 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.195543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.196569 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.196575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.197617 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.197629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.198426 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.198433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.198608 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.199483 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:44.199501 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:44.200745 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:44.205672 (+ 4927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:44.205676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:44.205679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:44.206343 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:44.206842 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:44.206847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:44.206854 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:44.207020 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:44.207614 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:44.207621 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:44.207855 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.208042 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.208566 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":379,"threads_started":1}
I20260807 19:43:44.217027 2355 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:33663
I20260807 19:43:44.217049 2495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:33663 every 8 connection(s)
I20260807 19:43:44.218356 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:44.220745 2497 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37481
I20260807 19:43:44.220997 2497 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:44.221668 2497 heartbeater.cc:511] Master 127.0.198.254:37481 requested a full tablet report, sending...
I20260807 19:43:44.223270 1985 ts_manager.cc:194] Registered new tserver with Master: 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:44.223510 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2355
I20260807 19:43:44.223732 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:44.224284 795 external_mini_cluster.cc:1428] Setting key f5a750eefd21c8ef4dbfa5e96f3c497c
I20260807 19:43:44.224524 1985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:46303
I20260807 19:43:44.227805 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:43:44 dist-test-slave-btw8 krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:43:44 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:44.289225 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.272868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.0.198.254:37481)
0807 19:43:44.273155 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.273163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.273349 (+ 186us) server_negotiation.cc:407] Connection header received
0807 19:43:44.273617 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.273623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.273697 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.273819 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:44.275469 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.276297 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.277742 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.278230 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.278389 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.282975 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:44.282981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:44.282985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:44.283027 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:44.286230 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.286959 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.286967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.286970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.287128 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:44.287748 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:44.287752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:44.287755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:44.288055 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:44.288302 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.288691 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.288928 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:43:44.293689 1985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:44.300812 1985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:44.340273 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.314419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:42809 (local address 127.0.0.1:56668)
0807 19:43:44.328255 (+ 13836us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.328339 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.329640 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.331023 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.331031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.331078 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:44.331642 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.331657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.333677 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.333684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.336429 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.336451 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.339046 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.339054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.339344 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.339452 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.339596 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.339812 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13409,"mutex_wait_us":1,"thread_start_us":22924,"threads_started":1}
I20260807 19:43:44.340299 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.315048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.0.198.193:42809)
0807 19:43:44.324359 (+ 9311us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.324376 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.328533 (+ 4157us) server_negotiation.cc:407] Connection header received
0807 19:43:44.330010 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.330024 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.330407 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.330625 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:44.331856 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.333491 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.336667 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.338469 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.338591 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.338827 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.339767 (+ 940us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.339978 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9022,"thread_start_us":8797,"threads_started":1}
I20260807 19:43:44.344379 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.315400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.0.198.195:33663)
0807 19:43:44.331220 (+ 15820us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.331231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.332608 (+ 1377us) server_negotiation.cc:407] Connection header received
0807 19:43:44.332725 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.332733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.333098 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.333289 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:44.334469 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.336290 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.340983 (+ 4693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.342562 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.342717 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.342969 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.343822 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.344004 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15613,"thread_start_us":15851,"threads_started":1}
I20260807 19:43:44.345161 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.314782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:33663 (local address 127.0.0.1:56614)
0807 19:43:44.332333 (+ 17551us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.332355 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.332480 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.333610 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.333616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.333659 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:44.334221 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.334235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.337188 (+ 2953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.337195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.340739 (+ 3544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.340760 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.343120 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.343127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.343417 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.343458 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.343570 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.344729 (+ 1159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17392,"mutex_wait_us":2231,"thread_start_us":13805,"threads_started":1}
I20260807 19:43:44.345247 2131 tablet_service.cc:1467] Processing CreateTablet for tablet dc5093b1e8094a3e868a6425b154e50c (DEFAULT_TABLE table=test-table [id=5807e1194a15453ab7200bdb6f370214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.347880 2131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5093b1e8094a3e868a6425b154e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.348248 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.314418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43231 (local address 127.0.0.1:39950)
0807 19:43:44.336912 (+ 22494us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.336934 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.337049 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.340460 (+ 3411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.340468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.340507 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:44.340924 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.340940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.343013 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.343020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.345329 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.345350 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.347287 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.347295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.347527 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.347571 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.347667 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.347831 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22346,"mutex_wait_us":2117,"thread_start_us":15474,"threads_started":1}
I20260807 19:43:44.349371 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.314657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.0.198.194:43231)
0807 19:43:44.339723 (+ 25066us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.339732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.339763 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:43:44.339880 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.339889 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.340184 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.341198 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:44.341425 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.342815 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.345585 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.347015 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.348489 (+ 1474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.348780 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.348862 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.349087 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24872,"thread_start_us":24736,"threads_started":1}
I20260807 19:43:44.349743 2424 tablet_service.cc:1467] Processing CreateTablet for tablet dc5093b1e8094a3e868a6425b154e50c (DEFAULT_TABLE table=test-table [id=5807e1194a15453ab7200bdb6f370214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.351490 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5093b1e8094a3e868a6425b154e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.354830 2277 tablet_service.cc:1467] Processing CreateTablet for tablet dc5093b1e8094a3e868a6425b154e50c (DEFAULT_TABLE table=test-table [id=5807e1194a15453ab7200bdb6f370214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.357129 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5093b1e8094a3e868a6425b154e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.363772 2517 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Bootstrap starting.
I20260807 19:43:44.367391 2518 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Bootstrap starting.
I20260807 19:43:44.368794 2517 tablet_bootstrap.cc:654] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.371166 2517 log.cc:824] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:44.371301 2518 tablet_bootstrap.cc:654] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.372643 2518 log.cc:824] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:44.374902 2517 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: No bootstrap required, opened a new log
I20260807 19:43:44.375059 2521 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Bootstrap starting.
I20260807 19:43:44.375216 2517 ts_tablet_manager.cc:1397] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.002s
I20260807 19:43:44.375638 2518 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: No bootstrap required, opened a new log
I20260807 19:43:44.375908 2518 ts_tablet_manager.cc:1397] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260807 19:43:44.379127 2521 tablet_bootstrap.cc:654] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.380630 2521 log.cc:824] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:44.385046 2521 tablet_bootstrap.cc:492] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: No bootstrap required, opened a new log
I20260807 19:43:44.384946 2518 raft_consensus.cc:348] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.385416 2521 ts_tablet_manager.cc:1397] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 19:43:44.385466 2518 raft_consensus.cc:374] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.385581 2518 raft_consensus.cc:729] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47646cfca741daa8db71b287cbea76, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.386251 2518 consensus_queue.cc:260] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.387499 2497 heartbeater.cc:503] Master 127.0.198.254:37481 was elected leader, sending a full tablet report...
I20260807 19:43:44.388172 2518 ts_tablet_manager.cc:1428] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260807 19:43:44.389667 2517 raft_consensus.cc:348] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.390053 2517 raft_consensus.cc:374] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.390182 2517 raft_consensus.cc:729] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b2e1919f27432d8518403715a061b2, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.390816 2517 consensus_queue.cc:260] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.392634 2517 ts_tablet_manager.cc:1428] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Time spent starting tablet: real 0.017s user 0.015s sys 0.001s
I20260807 19:43:44.397015 2521 raft_consensus.cc:348] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.397440 2521 raft_consensus.cc:374] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.397524 2521 raft_consensus.cc:729] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1fe2e357e34cc2a9beeab70f35e611, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.398145 2521 consensus_queue.cc:260] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.399250 2351 heartbeater.cc:503] Master 127.0.198.254:37481 was elected leader, sending a full tablet report...
I20260807 19:43:44.399863 2521 ts_tablet_manager.cc:1428] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
W20260807 19:43:44.470217 2498 tablet.cc:2369] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:43:44.555166 2206 tablet.cc:2369] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:43:44.646416 2352 tablet.cc:2369] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:44.703104 2525 raft_consensus.cc:482] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:44.703588 2525 raft_consensus.cc:504] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.705592 2525 leader_election.cc:288] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:44.713857 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.706191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:38957 (local address 127.0.198.194:43231)
0807 19:43:44.706440 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.706449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.707291 (+ 842us) server_negotiation.cc:407] Connection header received
0807 19:43:44.707621 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.707626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.707694 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.707811 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:44.708536 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.709436 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.711796 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.712679 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.712903 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.713004 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.713346 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.713494 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:43:44.713877 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.706234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:47449 (local address 127.0.198.195:33663)
0807 19:43:44.706452 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.706463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.707402 (+ 939us) server_negotiation.cc:407] Connection header received
0807 19:43:44.707637 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.707643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.707709 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.707817 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:44.708800 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.709800 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.711902 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.712751 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.712993 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.713093 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.713411 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.713562 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:43:44.714135 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.706334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:33663 (local address 127.0.198.193:47449)
0807 19:43:44.707233 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.707262 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.707460 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.707842 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.707847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.707897 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:44.708566 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.708580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.709949 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.709956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.711711 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.711725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.713030 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.713037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.713232 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.713259 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.713377 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.713526 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":1,"spinlock_wait_cycles":1920,"thread_start_us":558,"threads_started":1}
I20260807 19:43:44.714165 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.706194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43231 (local address 127.0.198.193:38957)
0807 19:43:44.707137 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:44.707159 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:44.707437 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:44.707831 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:44.707836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:44.707864 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:44.708353 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.708368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.709592 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.709599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.711603 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:44.711619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.712968 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.712975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:44.713157 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.713186 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:43:44.713313 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 19:43:44.713474 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":30,"thread_start_us":514,"threads_started":1}
I20260807 19:43:44.715341 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5093b1e8094a3e868a6425b154e50c" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76" is_pre_election: true
I20260807 19:43:44.715940 2448 raft_consensus.cc:2454] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 0.
I20260807 19:43:44.715811 2302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5093b1e8094a3e868a6425b154e50c" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" is_pre_election: true
I20260807 19:43:44.716398 2302 raft_consensus.cc:2454] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 0.
I20260807 19:43:44.716938 2078 leader_election.cc:302] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b2e1919f27432d8518403715a061b2, 4d47646cfca741daa8db71b287cbea76; no voters:
I20260807 19:43:44.717448 2525 raft_consensus.cc:2781] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:44.717615 2525 raft_consensus.cc:482] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:44.717789 2525 raft_consensus.cc:3037] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:44.720260 2525 raft_consensus.cc:504] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.720983 2525 leader_election.cc:288] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:44.721585 2302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5093b1e8094a3e868a6425b154e50c" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1fe2e357e34cc2a9beeab70f35e611"
I20260807 19:43:44.721807 2302 raft_consensus.cc:3037] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:44.721725 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5093b1e8094a3e868a6425b154e50c" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76"
I20260807 19:43:44.722014 2448 raft_consensus.cc:3037] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:44.724689 2302 raft_consensus.cc:2454] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 1.
I20260807 19:43:44.725003 2448 raft_consensus.cc:2454] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 1.
I20260807 19:43:44.725298 2081 leader_election.cc:302] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b2e1919f27432d8518403715a061b2, 4c1fe2e357e34cc2a9beeab70f35e611; no voters:
I20260807 19:43:44.725677 2525 raft_consensus.cc:2781] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:44.726183 2525 raft_consensus.cc:686] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 1 LEADER]: Becoming Leader. State: Replica: 31b2e1919f27432d8518403715a061b2, State: Running, Role: LEADER
I20260807 19:43:44.726832 2525 consensus_queue.cc:237] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.732192 1983 catalog_manager.cc:5495] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31b2e1919f27432d8518403715a061b2 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "31b2e1919f27432d8518403715a061b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } health_report { overall_health: UNKNOWN } } }
I20260807 19:43:44.834530 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:44.827748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.0.198.193:42809)
0807 19:43:44.828027 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:44.828035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:44.828465 (+ 430us) server_negotiation.cc:407] Connection header received
0807 19:43:44.828906 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:44.828910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:44.829063 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:44.829219 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:44.829712 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.830451 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.832048 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:44.832511 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:44.832654 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:44.833212 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:44.833796 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:44.833959 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:44.834099 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 19:43:44.834194 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:43:44.843957 2302 raft_consensus.cc:1261] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Refusing update from remote peer 31b2e1919f27432d8518403715a061b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:44.843956 2448 raft_consensus.cc:1261] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Refusing update from remote peer 31b2e1919f27432d8518403715a061b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:44.845139 2529 consensus_queue.cc:1035] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:44.845655 2525 consensus_queue.cc:1035] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:44.880846 1983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:44.881181 1983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:44.885721 1983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Sending DeleteTablet for 3 replicas of tablet dc5093b1e8094a3e868a6425b154e50c
I20260807 19:43:44.887099 2424 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5093b1e8094a3e868a6425b154e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:44 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56614
I20260807 19:43:44.887112 2277 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5093b1e8094a3e868a6425b154e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:44 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:39950
I20260807 19:43:44.887902 2131 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5093b1e8094a3e868a6425b154e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:44 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56668
I20260807 19:43:44.888258 2543 tablet_replica.cc:330] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: stopping tablet replica
I20260807 19:43:44.888749 2544 tablet_replica.cc:330] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: stopping tablet replica
I20260807 19:43:44.889271 2543 raft_consensus.cc:2229] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:44.889894 2543 raft_consensus.cc:2258] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:44.890748 1983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:44.891558 1983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:44.892899 2543 ts_tablet_manager.cc:1905] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:44.894106 2544 raft_consensus.cc:2229] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:44.894663 2544 raft_consensus.cc:2258] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:44.898444 2545 tablet_replica.cc:330] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: stopping tablet replica
I20260807 19:43:44.898945 2545 raft_consensus.cc:2229] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:44.899607 2545 raft_consensus.cc:2258] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:44.900532 2424 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0445c51f214668a3887aef62aebc72 (DEFAULT_TABLE table=test-table [id=11965985fb3d4d71abdab606abe43a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.900566 2130 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0445c51f214668a3887aef62aebc72 (DEFAULT_TABLE table=test-table [id=11965985fb3d4d71abdab606abe43a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.901033 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0445c51f214668a3887aef62aebc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.901044 2130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0445c51f214668a3887aef62aebc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.904047 2277 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0445c51f214668a3887aef62aebc72 (DEFAULT_TABLE table=test-table [id=11965985fb3d4d71abdab606abe43a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:44.904691 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0445c51f214668a3887aef62aebc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:44.905876 2543 ts_tablet_manager.cc:1918] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:44.907514 2543 log.cc:1196] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/dc5093b1e8094a3e868a6425b154e50c
I20260807 19:43:44.908315 2543 ts_tablet_manager.cc:1939] T dc5093b1e8094a3e868a6425b154e50c P 4d47646cfca741daa8db71b287cbea76: Deleting consensus metadata
I20260807 19:43:44.909680 2545 ts_tablet_manager.cc:1905] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:44.912415 1966 catalog_manager.cc:4841] TS 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663): tablet dc5093b1e8094a3e868a6425b154e50c (table test-table [id=5807e1194a15453ab7200bdb6f370214]) successfully deleted
I20260807 19:43:44.918427 2548 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Bootstrap starting.
I20260807 19:43:44.919919 2547 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Bootstrap starting.
I20260807 19:43:44.920737 2544 ts_tablet_manager.cc:1905] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:44.922096 2547 tablet_bootstrap.cc:654] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.922549 2545 ts_tablet_manager.cc:1918] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:44.922873 2545 log.cc:1196] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/dc5093b1e8094a3e868a6425b154e50c
I20260807 19:43:44.922946 2548 tablet_bootstrap.cc:654] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.923779 2545 ts_tablet_manager.cc:1939] T dc5093b1e8094a3e868a6425b154e50c P 31b2e1919f27432d8518403715a061b2: Deleting consensus metadata
I20260807 19:43:44.926122 1968 catalog_manager.cc:4841] TS 31b2e1919f27432d8518403715a061b2 (127.0.198.193:42809): tablet dc5093b1e8094a3e868a6425b154e50c (table test-table [id=5807e1194a15453ab7200bdb6f370214]) successfully deleted
I20260807 19:43:44.927170 2546 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Bootstrap starting.
I20260807 19:43:44.928930 2546 tablet_bootstrap.cc:654] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:44.929540 2544 ts_tablet_manager.cc:1918] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:44.929744 2544 log.cc:1196] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/dc5093b1e8094a3e868a6425b154e50c
I20260807 19:43:44.930297 2544 ts_tablet_manager.cc:1939] T dc5093b1e8094a3e868a6425b154e50c P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting consensus metadata
I20260807 19:43:44.932297 2548 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: No bootstrap required, opened a new log
I20260807 19:43:44.932504 2548 ts_tablet_manager.cc:1397] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260807 19:43:44.932801 1971 catalog_manager.cc:4841] TS 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231): tablet dc5093b1e8094a3e868a6425b154e50c (table test-table [id=5807e1194a15453ab7200bdb6f370214]) successfully deleted
I20260807 19:43:44.933346 2548 raft_consensus.cc:348] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.933568 2548 raft_consensus.cc:374] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.933658 2548 raft_consensus.cc:729] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1fe2e357e34cc2a9beeab70f35e611, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.933946 2548 consensus_queue.cc:260] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.934661 2548 ts_tablet_manager.cc:1428] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:43:44.934700 2547 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: No bootstrap required, opened a new log
I20260807 19:43:44.934870 2547 ts_tablet_manager.cc:1397] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260807 19:43:44.935927 2547 raft_consensus.cc:348] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.936228 2547 raft_consensus.cc:374] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.936329 2547 raft_consensus.cc:729] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b2e1919f27432d8518403715a061b2, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.936645 2547 consensus_queue.cc:260] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.937291 2547 ts_tablet_manager.cc:1428] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:43:44.938380 2546 tablet_bootstrap.cc:492] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: No bootstrap required, opened a new log
I20260807 19:43:44.938567 2546 ts_tablet_manager.cc:1397] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20260807 19:43:44.939438 2546 raft_consensus.cc:348] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.939749 2546 raft_consensus.cc:374] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:44.939910 2546 raft_consensus.cc:729] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47646cfca741daa8db71b287cbea76, State: Initialized, Role: FOLLOWER
I20260807 19:43:44.940224 2546 consensus_queue.cc:260] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:44.941015 2546 ts_tablet_manager.cc:1428] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 19:43:45.085675 2529 raft_consensus.cc:482] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:45.085958 2529 raft_consensus.cc:504] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.086774 2529 leader_election.cc:288] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:45.087563 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0445c51f214668a3887aef62aebc72" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76" is_pre_election: true
I20260807 19:43:45.087659 2301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0445c51f214668a3887aef62aebc72" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" is_pre_election: true
I20260807 19:43:45.087888 2448 raft_consensus.cc:2454] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 0.
I20260807 19:43:45.088006 2301 raft_consensus.cc:2454] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 0.
I20260807 19:43:45.088526 2081 leader_election.cc:302] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b2e1919f27432d8518403715a061b2, 4c1fe2e357e34cc2a9beeab70f35e611; no voters:
I20260807 19:43:45.088881 2529 raft_consensus.cc:2781] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:45.089020 2529 raft_consensus.cc:482] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:45.089146 2529 raft_consensus.cc:3037] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.091658 2529 raft_consensus.cc:504] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.092324 2529 leader_election.cc:288] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:45.092880 2301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0445c51f214668a3887aef62aebc72" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1fe2e357e34cc2a9beeab70f35e611"
I20260807 19:43:45.093060 2301 raft_consensus.cc:3037] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.092957 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0445c51f214668a3887aef62aebc72" candidate_uuid: "31b2e1919f27432d8518403715a061b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76"
I20260807 19:43:45.093180 2448 raft_consensus.cc:3037] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.095523 2301 raft_consensus.cc:2454] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 1.
I20260807 19:43:45.096027 2081 leader_election.cc:302] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b2e1919f27432d8518403715a061b2, 4c1fe2e357e34cc2a9beeab70f35e611; no voters:
I20260807 19:43:45.096288 2448 raft_consensus.cc:2454] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31b2e1919f27432d8518403715a061b2 in term 1.
I20260807 19:43:45.096328 2529 raft_consensus.cc:2781] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:45.096525 2529 raft_consensus.cc:686] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 1 LEADER]: Becoming Leader. State: Replica: 31b2e1919f27432d8518403715a061b2, State: Running, Role: LEADER
I20260807 19:43:45.096841 2529 consensus_queue.cc:237] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.100162 1983 catalog_manager.cc:5495] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31b2e1919f27432d8518403715a061b2 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "31b2e1919f27432d8518403715a061b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:43:45 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:45.162967 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.146232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37481 (local address 127.0.0.1:57744)
0807 19:43:45.147168 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.147186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.147427 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.147874 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.147878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.148534 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:45.148807 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.148817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.150364 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.150369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.151532 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.151545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.152554 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.152559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.152738 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.153501 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:45.153525 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:45.156170 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:45.160235 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:45.160258 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:45.160265 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:45.160703 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:45.161104 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:45.161110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:45.161113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:45.161265 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:45.161817 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:45.161826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:45.162004 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.162186 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.162504 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":658,"threads_started":1}
I20260807 19:43:45.163035 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.146229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.0.198.254:37481)
0807 19:43:45.147026 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.147034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.147284 (+ 250us) server_negotiation.cc:407] Connection header received
0807 19:43:45.147592 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.147599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.147685 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.147819 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.148972 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.150198 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.151685 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.152356 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.152491 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.156425 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.156434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.156439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.156484 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.160074 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.160846 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.160854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.160862 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.160991 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.161387 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.161394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.161401 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.161676 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.161859 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.162280 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.162662 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":605,"thread_start_us":359,"threads_started":1}
I20260807 19:43:45.170229 1983 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:57744:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:43:45.171537 1983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:45.179235 2130 tablet_service.cc:1467] Processing CreateTablet for tablet 28a468d93ab04a56ad7f61228a70240e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=344d0c70d296403ca509754956bb234c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:45.179651 2424 tablet_service.cc:1467] Processing CreateTablet for tablet 28a468d93ab04a56ad7f61228a70240e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=344d0c70d296403ca509754956bb234c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:45.179876 2130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a468d93ab04a56ad7f61228a70240e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:45.179796 2277 tablet_service.cc:1467] Processing CreateTablet for tablet 28a468d93ab04a56ad7f61228a70240e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=344d0c70d296403ca509754956bb234c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:45.180204 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a468d93ab04a56ad7f61228a70240e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:45.180328 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a468d93ab04a56ad7f61228a70240e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:45.185756 2547 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: Bootstrap starting.
I20260807 19:43:45.187709 2547 tablet_bootstrap.cc:654] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:45.187759 2546 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: Bootstrap starting.
I20260807 19:43:45.189781 2546 tablet_bootstrap.cc:654] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:45.191072 2548 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: Bootstrap starting.
I20260807 19:43:45.192991 2548 tablet_bootstrap.cc:654] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:45.193385 2547 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: No bootstrap required, opened a new log
I20260807 19:43:45.193578 2547 ts_tablet_manager.cc:1397] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 19:43:45.194567 2547 raft_consensus.cc:348] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.194866 2547 raft_consensus.cc:374] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:45.194986 2547 raft_consensus.cc:729] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b2e1919f27432d8518403715a061b2, State: Initialized, Role: FOLLOWER
I20260807 19:43:45.195317 2546 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: No bootstrap required, opened a new log
I20260807 19:43:45.195263 2547 consensus_queue.cc:260] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.195536 2546 ts_tablet_manager.cc:1397] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 19:43:45.195876 2529 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.196220 2529 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.196907 2548 tablet_bootstrap.cc:492] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: No bootstrap required, opened a new log
I20260807 19:43:45.197081 2548 ts_tablet_manager.cc:1397] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260807 19:43:45.197520 2546 raft_consensus.cc:348] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.197805 2546 raft_consensus.cc:374] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:45.197880 2546 raft_consensus.cc:729] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47646cfca741daa8db71b287cbea76, State: Initialized, Role: FOLLOWER
I20260807 19:43:45.197881 2548 raft_consensus.cc:348] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.198163 2548 raft_consensus.cc:374] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:45.198246 2548 raft_consensus.cc:729] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1fe2e357e34cc2a9beeab70f35e611, State: Initialized, Role: FOLLOWER
I20260807 19:43:45.198191 2546 consensus_queue.cc:260] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.198725 2548 consensus_queue.cc:260] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.198764 2524 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.199159 2524 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.199370 2547 ts_tablet_manager.cc:1428] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 19:43:45.199259 2526 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.199666 2526 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.200852 2546 ts_tablet_manager.cc:1428] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:43:45.202307 2548 ts_tablet_manager.cc:1428] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260807 19:43:45.481797 2526 raft_consensus.cc:482] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:45.482064 2526 raft_consensus.cc:504] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.483968 2526 leader_election.cc:288] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31b2e1919f27432d8518403715a061b2 (127.0.198.193:42809), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:45.491317 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.484620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:45853 (local address 127.0.198.193:42809)
0807 19:43:45.485740 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.485750 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.485776 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:43:45.485877 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.485885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.485990 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.486130 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:45.486805 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.487745 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.489538 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.490334 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.490535 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.490613 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.490869 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.491014 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":826,"threads_started":1}
I20260807 19:43:45.491381 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.484148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:42809 (local address 127.0.198.194:45853)
0807 19:43:45.484886 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.484915 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.485141 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.486182 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.486188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.486226 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:45.486609 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.486627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.487902 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.487910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.489360 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.489373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.490521 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.490526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.490703 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.490728 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.490838 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.490984 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":630,"threads_started":1}
I20260807 19:43:45.492208 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.484239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:33663 (local address 127.0.198.194:55279)
0807 19:43:45.486740 (+ 2501us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.486767 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.486893 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.487224 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.487229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.487258 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:45.487657 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.487667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.488811 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.488817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.490189 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.490202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.491374 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.491382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.491574 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.491609 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.491691 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.491827 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2369,"thread_start_us":2246,"threads_started":1}
I20260807 19:43:45.492650 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.484640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:55279 (local address 127.0.198.195:33663)
0807 19:43:45.485977 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.485986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.486928 (+ 942us) server_negotiation.cc:407] Connection header received
0807 19:43:45.487011 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.487014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.487078 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.487224 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:45.487830 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.488668 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.490352 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.491135 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.491333 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.491453 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.492172 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.492356 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1173,"thread_start_us":1075,"threads_started":1}
I20260807 19:43:45.492501 2158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a468d93ab04a56ad7f61228a70240e" candidate_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31b2e1919f27432d8518403715a061b2" is_pre_election: true
I20260807 19:43:45.492938 2158 raft_consensus.cc:2454] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c1fe2e357e34cc2a9beeab70f35e611 in term 0.
I20260807 19:43:45.493327 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a468d93ab04a56ad7f61228a70240e" candidate_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76" is_pre_election: true
I20260807 19:43:45.493645 2448 raft_consensus.cc:2454] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c1fe2e357e34cc2a9beeab70f35e611 in term 0.
I20260807 19:43:45.493741 2226 leader_election.cc:302] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b2e1919f27432d8518403715a061b2, 4c1fe2e357e34cc2a9beeab70f35e611; no voters:
I20260807 19:43:45.494243 2526 raft_consensus.cc:2781] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:45.494402 2526 raft_consensus.cc:482] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:45.494518 2526 raft_consensus.cc:3037] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.496757 2526 raft_consensus.cc:504] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.497406 2526 leader_election.cc:288] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [CANDIDATE]: Term 1 election: Requested vote from peers 31b2e1919f27432d8518403715a061b2 (127.0.198.193:42809), 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663)
I20260807 19:43:45.497959 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a468d93ab04a56ad7f61228a70240e" candidate_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47646cfca741daa8db71b287cbea76"
I20260807 19:43:45.498034 2158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a468d93ab04a56ad7f61228a70240e" candidate_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31b2e1919f27432d8518403715a061b2"
I20260807 19:43:45.498200 2448 raft_consensus.cc:3037] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.498315 2158 raft_consensus.cc:3037] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:45.500417 2448 raft_consensus.cc:2454] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c1fe2e357e34cc2a9beeab70f35e611 in term 1.
I20260807 19:43:45.500667 2158 raft_consensus.cc:2454] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c1fe2e357e34cc2a9beeab70f35e611 in term 1.
I20260807 19:43:45.501016 2225 leader_election.cc:302] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1fe2e357e34cc2a9beeab70f35e611, 4d47646cfca741daa8db71b287cbea76; no voters:
I20260807 19:43:45.501384 2526 raft_consensus.cc:2781] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:45.501842 2526 raft_consensus.cc:686] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 LEADER]: Becoming Leader. State: Replica: 4c1fe2e357e34cc2a9beeab70f35e611, State: Running, Role: LEADER
I20260807 19:43:45.502239 2526 consensus_queue.cc:237] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } }
I20260807 19:43:45.504258 2569 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: TxnStatusTablet state changed. Reason: New leader 4c1fe2e357e34cc2a9beeab70f35e611. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.504709 2569 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:45.506376 2571 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:43:45.506699 2571 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:43:45.506314 1983 catalog_manager.cc:5495] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } health_report { overall_health: UNKNOWN } } }
I20260807 19:43:45.574110 2529 consensus_queue.cc:1035] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:45.587859 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.568737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43231 (local address 127.0.0.1:34436)
0807 19:43:45.569075 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.569104 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.569297 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.576711 (+ 7414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.576746 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.576765 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:45.577022 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.577037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.581738 (+ 4701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.581746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.583625 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.583648 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.585107 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.585116 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.585418 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.585535 (+ 117us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.587056 (+ 1521us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.587094 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.587197 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.587442 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:43:45.587987 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.568669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.0.198.194:43231)
0807 19:43:45.576037 (+ 7368us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.576045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.576086 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:45.576223 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.576229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.576351 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.577285 (+ 934us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:45.577630 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.581457 (+ 3827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.583809 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.584865 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.585062 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.585840 (+ 778us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.586906 (+ 1066us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.587126 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.587264 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.587430 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7192,"thread_start_us":6941,"threads_started":1}
I20260807 19:43:45.594707 2448 raft_consensus.cc:1261] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Refusing update from remote peer 4c1fe2e357e34cc2a9beeab70f35e611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:45.595384 2529 consensus_queue.cc:1035] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:45.595880 2158 raft_consensus.cc:1261] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Refusing update from remote peer 4c1fe2e357e34cc2a9beeab70f35e611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:45.595757 2569 consensus_queue.cc:1035] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:45.596768 2569 consensus_queue.cc:1035] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:45.608773 2524 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: TxnStatusTablet state changed. Reason: New leader 4c1fe2e357e34cc2a9beeab70f35e611. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.609160 2524 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.616488 2578 mvcc.cc:205] Tried to move back new op lower bound from 7315995957626511360 to 7315995957260636160. Current Snapshot: MvccSnapshot[applied={T|T < 7315995957626511360}]
I20260807 19:43:45.617192 2525 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: TxnStatusTablet state changed. Reason: New leader 4c1fe2e357e34cc2a9beeab70f35e611. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.617519 2525 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.621152 2524 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.621626 2524 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4d47646cfca741daa8db71b287cbea76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.625813 2576 mvcc.cc:205] Tried to move back new op lower bound from 7315995957626511360 to 7315995957260636160. Current Snapshot: MvccSnapshot[applied={T|T < 7315995957626511360}]
I20260807 19:43:45.627333 2526 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.627945 2526 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:45.628409 2526 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.628814 2526 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 4c1fe2e357e34cc2a9beeab70f35e611: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:45.630321 2525 tablet_replica.cc:439] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31b2e1919f27432d8518403715a061b2" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42809 } } peers { permanent_uuid: "4c1fe2e357e34cc2a9beeab70f35e611" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43231 } } peers { permanent_uuid: "4d47646cfca741daa8db71b287cbea76" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 33663 } } }
I20260807 19:43:45.630625 2525 tablet_replica.cc:442] T 28a468d93ab04a56ad7f61228a70240e P 31b2e1919f27432d8518403715a061b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:45.655668 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.646914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:47081 (local address 127.0.198.194:43231)
0807 19:43:45.647130 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.647137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.648111 (+ 974us) server_negotiation.cc:407] Connection header received
0807 19:43:45.648340 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.648346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.648436 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.648498 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:45.649356 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.650764 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.653042 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.653890 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.654052 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.654660 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.654819 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.654928 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.655185 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.655305 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:43:45.655682 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.646922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43231 (local address 127.0.198.193:47081)
0807 19:43:45.647944 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.647971 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.648154 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.648709 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.648714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.648727 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:45.649161 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.649173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.650997 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.651004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.652830 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.652852 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.654131 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.654138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.654335 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.654431 (+ 96us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.654950 (+ 519us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.654962 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.655046 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.655213 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":620,"threads_started":1}
I20260807 19:43:45.708583 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.701983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:60947 (local address 127.0.198.193:42809)
0807 19:43:45.702195 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.702202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.702978 (+ 776us) server_negotiation.cc:407] Connection header received
0807 19:43:45.703394 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.703402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.703509 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.703628 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:45.704253 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.704977 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.706456 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.706912 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.707058 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.707656 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.707830 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.707928 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.708136 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.708226 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:43:45.708702 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.701826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:42809 (local address 127.0.198.194:60947)
0807 19:43:45.702850 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:45.702867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:45.703116 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:45.703813 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:45.703818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:45.703831 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:45.704066 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.704083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.705131 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.705137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.706298 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:45.706311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.707074 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.707079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:45.707306 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.707414 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:45.707950 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:45.707962 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:43:45.708104 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:43:45.708277 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":639,"threads_started":1}
I20260807 19:43:45.754017 1981 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:45.754320 1981 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:45.758157 1981 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4af3385d5d1f4b5fa9c72072de2598e2: Sending DeleteTablet for 3 replicas of tablet 9a0445c51f214668a3887aef62aebc72
I20260807 19:43:45.759326 2424 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0445c51f214668a3887aef62aebc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:45 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56614
I20260807 19:43:45.759473 2277 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0445c51f214668a3887aef62aebc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:45 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:39950
I20260807 19:43:45.759485 2130 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0445c51f214668a3887aef62aebc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:45 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56668
I20260807 19:43:45.771867 2598 tablet_replica.cc:330] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: stopping tablet replica
I20260807 19:43:45.772234 2598 raft_consensus.cc:2229] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:45.772605 2598 raft_consensus.cc:2258] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:45.774319 2598 ts_tablet_manager.cc:1905] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:45.774829 2600 tablet_replica.cc:330] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: stopping tablet replica
I20260807 19:43:45.775079 2600 raft_consensus.cc:2229] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:45.775743 2600 raft_consensus.cc:2258] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:45.776980 2600 ts_tablet_manager.cc:1905] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:45.777289 2599 tablet_replica.cc:330] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: stopping tablet replica
I20260807 19:43:45.777503 2599 raft_consensus.cc:2229] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:45.777812 2599 raft_consensus.cc:2258] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:45.785130 2600 ts_tablet_manager.cc:1918] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:45.785398 2600 log.cc:1196] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/9a0445c51f214668a3887aef62aebc72
I20260807 19:43:45.785889 2600 ts_tablet_manager.cc:1939] T 9a0445c51f214668a3887aef62aebc72 P 31b2e1919f27432d8518403715a061b2: Deleting consensus metadata
I20260807 19:43:45.785926 2598 ts_tablet_manager.cc:1918] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:45.786126 2598 log.cc:1196] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/9a0445c51f214668a3887aef62aebc72
I20260807 19:43:45.786810 2598 ts_tablet_manager.cc:1939] T 9a0445c51f214668a3887aef62aebc72 P 4d47646cfca741daa8db71b287cbea76: Deleting consensus metadata
I20260807 19:43:45.787168 1968 catalog_manager.cc:4841] TS 31b2e1919f27432d8518403715a061b2 (127.0.198.193:42809): tablet 9a0445c51f214668a3887aef62aebc72 (table test-table [id=11965985fb3d4d71abdab606abe43a6c]) successfully deleted
I20260807 19:43:45.788313 1966 catalog_manager.cc:4841] TS 4d47646cfca741daa8db71b287cbea76 (127.0.198.195:33663): tablet 9a0445c51f214668a3887aef62aebc72 (table test-table [id=11965985fb3d4d71abdab606abe43a6c]) successfully deleted
I20260807 19:43:45.788463 2599 ts_tablet_manager.cc:1905] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:45.796989 2599 ts_tablet_manager.cc:1918] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:45.797168 2599 log.cc:1196] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/9a0445c51f214668a3887aef62aebc72
I20260807 19:43:45.797693 2599 ts_tablet_manager.cc:1939] T 9a0445c51f214668a3887aef62aebc72 P 4c1fe2e357e34cc2a9beeab70f35e611: Deleting consensus metadata
Aug 07 19:43:45 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:45.798856 1971 catalog_manager.cc:4841] TS 4c1fe2e357e34cc2a9beeab70f35e611 (127.0.198.194:43231): tablet 9a0445c51f214668a3887aef62aebc72 (table test-table [id=11965985fb3d4d71abdab606abe43a6c]) successfully deleted
I20260807 19:43:45.805428 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.789258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.0.198.193:42809)
0807 19:43:45.789469 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.789475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.789564 (+ 89us) server_negotiation.cc:407] Connection header received
0807 19:43:45.789805 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.789811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.789896 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.790017 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.791457 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.792207 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.793738 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.794280 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.794437 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.798755 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.798778 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.798799 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.798848 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.802106 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.803055 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.803064 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.803066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.803238 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.803747 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.803751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.803753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.804145 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.804460 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.804835 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.805084 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:43:45.806267 2199 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48816
I20260807 19:43:45.830717 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.809930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.0.198.254:37481)
0807 19:43:45.814103 (+ 4173us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.814111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.814135 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:43:45.814210 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.814216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.814311 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.814448 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.816422 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.817681 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.819300 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.819974 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.820136 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.822683 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.822692 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.822697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.822745 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.826906 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.828050 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.828057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.828061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.828286 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.828767 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.828772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.828775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.829198 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.829474 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.829954 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.830321 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4011,"thread_start_us":3759,"threads_started":1}
W20260807 19:43:45.831580 1981 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57746
I20260807 19:43:45.834863 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2062
I20260807 19:43:45.856392 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.835928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.0.198.193:42809)
0807 19:43:45.836221 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.836231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.840084 (+ 3853us) server_negotiation.cc:407] Connection header received
0807 19:43:45.840320 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.840325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.840401 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.840496 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.842513 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.843415 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.845630 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.846104 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.846225 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.849832 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.849838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.849842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.849886 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.852978 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.854024 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.854031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.854033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.854228 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.854772 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.854777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.854779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.855254 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.855674 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.855945 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.856142 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 19:43:45.857321 2199 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48820
W20260807 19:43:45.858119 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2062: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:45.858392 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2062
I20260807 19:43:45.889533 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2209
Aug 07 19:43:45 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:43:45.910040 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.890448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.0.198.194:43231)
0807 19:43:45.890668 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.890674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.890696 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:43:45.890904 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.890911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.891027 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.891160 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.892454 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.893735 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.895449 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.896133 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.896291 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.901041 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.901057 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.901084 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.901133 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.905925 (+ 4792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.907065 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.907075 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.907096 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.907344 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.907935 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.907941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.907945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.908458 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.908906 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.909180 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.909432 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:43:45.910974 2345 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34438
W20260807 19:43:45.911741 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2209: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:45.911939 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2209
I20260807 19:43:45.939898 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2355
Aug 07 19:43:45 dist-test-slave-btw8 krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:43:45.958827 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:45.940788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.0.198.195:33663)
0807 19:43:45.941014 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:45.941022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:45.941048 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:43:45.941579 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:45.941584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:45.941685 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:45.941813 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:45.943799 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.944580 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.945920 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:45.946370 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:45.946518 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:45.951438 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:45.951458 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:45.951475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:45.951521 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:45.955321 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.956362 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.956371 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.956373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.956532 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:45.956999 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:45.957004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:45.957006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:45.957479 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:45.957763 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:45.958223 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 19:43:45.958479 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 19:43:45.959657 2491 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36366
W20260807 19:43:45.960497 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2355: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:45.960671 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2355
I20260807 19:43:45.981292 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 1949
I20260807 19:43:45.982579 2030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:46.105549 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 1949
2026-08-07T19:43:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5677 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:46 dist-test-slave-btw8 krb5kdc[2611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:46 dist-test-slave-btw8 krb5kdc[2611](info): set up 2 sockets
Aug 07 19:43:46 dist-test-slave-btw8 krb5kdc[2611](info): commencing operation
krb5kdc: starting...
W20260807 19:43:47.281736 795 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:43:47 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:43:47.548527 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33337
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:47.915143 2624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:47.915591 2624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:47.915771 2624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:47.923501 2624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:47.923658 2624 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:43:47.923741 2624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:47.923840 2624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:47.923934 2624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:47.924045 2624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:47.934973 2624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:47.936513 2624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:47.937947 2624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:47.951527 2629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:47.951471 2630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:47.951443 2632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:47.952411 2624 server_base.cc:1034] running on GCE node
I20260807 19:43:47.953323 2624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:47.954938 2624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:47.956189 2624 hybrid_clock.cc:648] HybridClock initialized: now 1786131827956116 us; error 48 us; skew 500 ppm
I20260807 19:43:47.956584 2624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:47 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:47.960417 2624 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:43:47.962369 2624 webserver.cc:460] Webserver started at http://127.0.198.254:41069/ using document root <none> and password file <none>
I20260807 19:43:47.963187 2624 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:47.963320 2624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:47.963739 2624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:47.967224 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "c2b2d7d625234fce803ede7a542f29df"
format_stamp: "Formatted at 2026-08-07 19:43:47 on dist-test-slave-btw8"
server_key: "42c04ef8426f828d5f5c32f4b4aae82f"
server_key_iv: "12de9b1af39e6e13606b24df8d91d9b8"
server_key_version: "encryptionkey@0"
I20260807 19:43:47.968233 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "c2b2d7d625234fce803ede7a542f29df"
format_stamp: "Formatted at 2026-08-07 19:43:47 on dist-test-slave-btw8"
server_key: "42c04ef8426f828d5f5c32f4b4aae82f"
server_key_iv: "12de9b1af39e6e13606b24df8d91d9b8"
server_key_version: "encryptionkey@0"
I20260807 19:43:47.974149 2624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:43:47.978027 2639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:47.979588 2624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:43:47.979847 2624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "c2b2d7d625234fce803ede7a542f29df"
format_stamp: "Formatted at 2026-08-07 19:43:47 on dist-test-slave-btw8"
server_key: "42c04ef8426f828d5f5c32f4b4aae82f"
server_key_iv: "12de9b1af39e6e13606b24df8d91d9b8"
server_key_version: "encryptionkey@0"
I20260807 19:43:47.980109 2624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:48.000470 2624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:48.005129 2624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:48.005455 2624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:48.031695 2624 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33337
I20260807 19:43:48.031759 2705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33337 every 8 connection(s)
I20260807 19:43:48.033370 2624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:48.036069 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2624
I20260807 19:43:48.036342 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:43:48.037109 795 external_mini_cluster.cc:1428] Setting key 68ea64d26845a8a7757618de9e80c205
I20260807 19:43:48.037972 2706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:48.053579 2706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Bootstrap starting.
Aug 07 19:43:48 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:48.057437 2706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:48.059003 2706 log.cc:824] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:48.062685 2706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: No bootstrap required, opened a new log
I20260807 19:43:48.066040 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:48.039576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.0.198.254:33337)
0807 19:43:48.040577 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:48.040593 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:48.040652 (+ 59us) server_negotiation.cc:407] Connection header received
0807 19:43:48.043313 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:48.043346 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:48.043966 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:48.044478 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:48.046162 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.048221 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.049952 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.051218 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.051519 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:48.056459 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:48.056474 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:48.056494 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:48.056553 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:48.060831 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.061663 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.061674 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.061685 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.061923 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.062428 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.062432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.062435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.063180 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:48.063539 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:48.064035 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 19:43:48.064537 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":466,"threads_started":1}
I20260807 19:43:48.071758 2706 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } }
I20260807 19:43:48.072141 2706 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:48.072219 2706 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b2d7d625234fce803ede7a542f29df, State: Initialized, Role: FOLLOWER
I20260807 19:43:48.072737 2706 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } }
I20260807 19:43:48.072943 2706 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:48.073107 2706 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:48.073302 2706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:48.075717 2706 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } }
I20260807 19:43:48.076303 2706 leader_election.cc:302] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b2d7d625234fce803ede7a542f29df; no voters:
I20260807 19:43:48.076967 2706 leader_election.cc:288] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:48.077152 2711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:48.078374 2711 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [term 1 LEADER]: Becoming Leader. State: Replica: c2b2d7d625234fce803ede7a542f29df, State: Running, Role: LEADER
I20260807 19:43:48.078941 2711 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } }
I20260807 19:43:48.079538 2706 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:48.082046 2712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2b2d7d625234fce803ede7a542f29df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } } }
I20260807 19:43:48.082098 2713 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b2d7d625234fce803ede7a542f29df. Latest consensus state: current_term: 1 leader_uuid: "c2b2d7d625234fce803ede7a542f29df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b2d7d625234fce803ede7a542f29df" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33337 } } }
I20260807 19:43:48.082559 2712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:48.082657 2713 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:48.086130 2720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:48.092140 2720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:48.102030 2720 catalog_manager.cc:1347] Generated new cluster ID: b153480e5a8b4e7f8225957fffb496ed
I20260807 19:43:48.102164 2720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:48.123799 2720 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:48.125582 2720 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:48.134730 2720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Generated new TSK 0
I20260807 19:43:48.135869 2720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:43:48.204470 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33337
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:48.567659 2733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:48.568015 2733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:48.568199 2733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:48.577304 2733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:48.577471 2733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:48.577642 2733 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:43:48.577732 2733 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:48.577845 2733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:43:48.589174 2733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:48.590728 2733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:48.592203 2733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:48.606849 2739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:48.606976 2740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:48.606932 2742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:48.607383 2733 server_base.cc:1034] running on GCE node
I20260807 19:43:48.608255 2733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:48.609346 2733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:48.610601 2733 hybrid_clock.cc:648] HybridClock initialized: now 1786131828610530 us; error 49 us; skew 500 ppm
I20260807 19:43:48.610980 2733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:48 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:48.621402 2733 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:43:48.623327 2733 webserver.cc:460] Webserver started at http://127.0.198.193:39253/ using document root <none> and password file <none>
I20260807 19:43:48.624195 2733 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:48.624325 2733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:48.624640 2733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:48.628269 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "0511bae4536d4286bb89b2552e693b16"
format_stamp: "Formatted at 2026-08-07 19:43:48 on dist-test-slave-btw8"
server_key: "fadcf50e41dd60d294c2f59e2b37d1e6"
server_key_iv: "16ea4224acfcff6f694b16b86bd2da0e"
server_key_version: "encryptionkey@0"
I20260807 19:43:48.629212 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "0511bae4536d4286bb89b2552e693b16"
format_stamp: "Formatted at 2026-08-07 19:43:48 on dist-test-slave-btw8"
server_key: "fadcf50e41dd60d294c2f59e2b37d1e6"
server_key_iv: "16ea4224acfcff6f694b16b86bd2da0e"
server_key_version: "encryptionkey@0"
I20260807 19:43:48.636046 2733 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:43:48.640393 2750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:48.642025 2733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:43:48.642292 2733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "0511bae4536d4286bb89b2552e693b16"
format_stamp: "Formatted at 2026-08-07 19:43:48 on dist-test-slave-btw8"
server_key: "fadcf50e41dd60d294c2f59e2b37d1e6"
server_key_iv: "16ea4224acfcff6f694b16b86bd2da0e"
server_key_version: "encryptionkey@0"
I20260807 19:43:48.642562 2733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:48.665865 2733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:48.670661 2733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:48.671036 2733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:48.674022 2733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:48.674214 2733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:48.674358 2733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:48.674476 2733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:48 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:48.701921 2760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:48.675380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:42955 (local address 127.0.198.254:33337)
0807 19:43:48.675954 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:48.675963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:48.677428 (+ 1465us) server_negotiation.cc:407] Connection header received
0807 19:43:48.680321 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:48.680330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:48.680516 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:48.680664 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:48.683537 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.685124 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.687454 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.688112 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.688272 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:48.692957 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:48.692969 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:48.692977 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:48.693037 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:48.697297 (+ 4260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.698632 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.698640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.698644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.698823 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.699449 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.699454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.699456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.699843 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:48.700042 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:48.701199 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0807 19:43:48.701487 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":292,"threads_started":1}
I20260807 19:43:48.703613 2761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:48.676125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.193:42955)
0807 19:43:48.677193 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:48.677251 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:48.679813 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:48.680985 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:48.680999 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:48.682039 (+ 1040us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:48.683237 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:48.683277 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.685440 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.685451 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:48.687179 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:48.687195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.688432 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.688441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:48.688710 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:48.689576 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:48.689604 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:48.692606 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:48.697588 (+ 4982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:48.697601 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:48.697636 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:48.698447 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:48.698966 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:48.698972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:48.698978 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:48.699237 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:48.700144 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:48.700157 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:48.700671 (+ 514us) client_negotiation.cc:769] Sending connection context
0807 19:43:48.701056 (+ 385us) client_negotiation.cc:240] Negotiation successful
0807 19:43:48.701755 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":493,"threads_started":1}
I20260807 19:43:48.728117 2760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:48.712890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:39201 (local address 127.0.198.254:33337)
0807 19:43:48.713134 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:48.713143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:48.713252 (+ 109us) server_negotiation.cc:407] Connection header received
0807 19:43:48.713495 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:48.713499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:48.713568 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:48.713691 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:48.714752 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.715570 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.718057 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.718516 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.718704 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:48.720950 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:48.720963 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:48.720966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:48.721004 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:48.724995 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.725841 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.725847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.725851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.726007 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:48.726442 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:48.726447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:48.726451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:48.726777 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:48.726951 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:48.727453 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 19:43:48.727719 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:43:48.728338 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:48.712351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.193:39201)
0807 19:43:48.713122 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:48.713145 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:48.713351 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:48.713757 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:48.713762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:48.714278 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:48.714575 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:48.714585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.715729 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.715737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:48.717865 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:48.717883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:48.718753 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:48.718760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:48.718939 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:48.719737 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:48.719759 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:48.720696 (+ 937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:48.725189 (+ 4493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:48.725195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:48.725199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:48.725684 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:48.726154 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:48.726159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:48.726163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:48.726317 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:48.726913 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:48.726918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:48.727137 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 19:43:48.727298 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:43:48.727774 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":610,"threads_started":1}
I20260807 19:43:48.734046 2733 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:38953
I20260807 19:43:48.734155 2873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:38953 every 8 connection(s)
I20260807 19:43:48.735522 2733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:48.738143 2874 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33337
I20260807 19:43:48.738480 2874 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:48.739282 2874 heartbeater.cc:511] Master 127.0.198.254:33337 requested a full tablet report, sending...
I20260807 19:43:48.741788 2656 ts_manager.cc:194] Registered new tserver with Master: 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953)
I20260807 19:43:48.743347 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2733
I20260807 19:43:48.743458 2656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:42955
I20260807 19:43:48.743739 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:48.744450 795 external_mini_cluster.cc:1428] Setting key d0f6df246bf74af8bee8dfb4011dfbcc
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:43:48.804418 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33337
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:49.173519 2878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:49.173874 2878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:49.173996 2878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:49.181829 2878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:49.181979 2878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:49.182086 2878 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:43:49.182148 2878 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:49.182220 2878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:43:49.192957 2878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:49.194355 2878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:49.195820 2878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:49.211503 2886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:49.211448 2884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:49.211555 2883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:49.212836 2878 server_base.cc:1034] running on GCE node
I20260807 19:43:49.213717 2878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:49.214900 2878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:49.216216 2878 hybrid_clock.cc:648] HybridClock initialized: now 1786131829216182 us; error 67 us; skew 500 ppm
I20260807 19:43:49.216646 2878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:49 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:49.220392 2878 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:43:49.222126 2878 webserver.cc:460] Webserver started at http://127.0.198.194:41093/ using document root <none> and password file <none>
I20260807 19:43:49.222875 2878 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:49.222988 2878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:49.223289 2878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:49.226724 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "3c38bb9708a04807881f1ac58062ba30"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "e8f4e6858df3faa5a5c73cdf2a5e70ec"
server_key_iv: "7fe1aa76d073e646068a879acbcbd6d4"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.227746 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "3c38bb9708a04807881f1ac58062ba30"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "e8f4e6858df3faa5a5c73cdf2a5e70ec"
server_key_iv: "7fe1aa76d073e646068a879acbcbd6d4"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.234153 2878 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260807 19:43:49.238584 2894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:49.240851 2878 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 19:43:49.241109 2878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "3c38bb9708a04807881f1ac58062ba30"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "e8f4e6858df3faa5a5c73cdf2a5e70ec"
server_key_iv: "7fe1aa76d073e646068a879acbcbd6d4"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.241369 2878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:49.255131 2878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:49.260352 2878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:49.260722 2878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:49.264394 2878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:49.264575 2878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:49.264750 2878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:49.264863 2878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:49 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:49.294308 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.265055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:49557 (local address 127.0.198.254:33337)
0807 19:43:49.265961 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:49.265967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:49.267199 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 19:43:49.270131 (+ 2932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:49.270138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:49.270254 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:49.270397 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:49.273411 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.274726 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.277639 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.278276 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.278449 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.285572 (+ 7123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:49.285582 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:49.285589 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:49.285649 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:49.288745 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.290342 (+ 1597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.290349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.290355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.290553 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.291392 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.291399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.291404 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.291943 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:49.292131 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:49.293502 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0807 19:43:49.293822 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":684,"threads_started":1}
I20260807 19:43:49.296708 2905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.265937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.194:49557)
0807 19:43:49.266979 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:49.267030 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:49.269734 (+ 2704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:49.270747 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:49.270765 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:49.271941 (+ 1176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:49.273132 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.273174 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.274908 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.274914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.277450 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.277465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.278849 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.278861 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.279194 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.280347 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:49.280373 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:49.285207 (+ 4834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:49.288958 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.288980 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.289009 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.290140 (+ 1131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.290729 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.290735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.290740 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.291193 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.292131 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:49.292144 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:49.292877 (+ 733us) client_negotiation.cc:769] Sending connection context
0807 19:43:49.293341 (+ 464us) client_negotiation.cc:240] Negotiation successful
0807 19:43:49.294190 (+ 849us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":471,"threads_started":1}
I20260807 19:43:49.314531 3021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:38957 every 8 connection(s)
I20260807 19:43:49.314530 2878 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:38957
I20260807 19:43:49.316798 2878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:49.319870 3022 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33337
I20260807 19:43:49.320181 3022 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:49.321221 3022 heartbeater.cc:511] Master 127.0.198.254:33337 requested a full tablet report, sending...
I20260807 19:43:49.322161 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 2878
I20260807 19:43:49.322359 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:49.322986 795 external_mini_cluster.cc:1428] Setting key c2deccafa7d9d08f8fed16f500745ac6
I20260807 19:43:49.323262 2656 ts_manager.cc:194] Registered new tserver with Master: 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957)
I20260807 19:43:49.324615 2656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:49557
I20260807 19:43:49.325853 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.305874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.194:55273)
0807 19:43:49.308063 (+ 2189us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:49.308087 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:49.308248 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:49.308697 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:49.308708 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:49.309182 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:49.309473 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.309483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.311429 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.311441 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.313562 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.313588 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.315185 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.315192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.315429 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.316205 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:49.316233 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:49.317574 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:49.322441 (+ 4867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.322448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.322452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.322967 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.323539 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.323544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.323547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.323719 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.324578 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:49.324585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:49.324802 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 19:43:49.324938 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 19:43:49.325374 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2029,"thread_start_us":1856,"threads_started":1}
I20260807 19:43:49.327400 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.306252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:55273 (local address 127.0.198.254:33337)
0807 19:43:49.306519 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:49.306529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:49.308228 (+ 1699us) server_negotiation.cc:407] Connection header received
0807 19:43:49.308385 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:49.308395 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:49.308512 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:49.308669 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:49.309662 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.311113 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.313854 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.314709 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.314867 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.317909 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:49.317918 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:49.317923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:49.317974 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:49.322189 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.323169 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.323176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.323180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.323407 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.323876 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.323884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.323888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.324419 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:49.326449 (+ 2030us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:49.326625 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 19:43:49.326922 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:43:49.383029 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33337
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:49.747498 2874 heartbeater.cc:503] Master 127.0.198.254:33337 was elected leader, sending a full tablet report...
W20260807 19:43:49.757628 3027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:49.757963 3027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:49.758077 3027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:49.766215 3027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:49.766362 3027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:49.766453 3027 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:43:49.766510 3027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:43:49.766592 3027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:43:49.777261 3027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:49.778641 3027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:49.780143 3027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:49.795259 3033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:49.795321 3037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:49.795297 3034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:49.796496 3027 server_base.cc:1034] running on GCE node
I20260807 19:43:49.797256 3027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:49.798650 3027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:49.799906 3027 hybrid_clock.cc:648] HybridClock initialized: now 1786131829799842 us; error 54 us; skew 500 ppm
I20260807 19:43:49.800387 3027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:49 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:49.804383 3027 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:43:49.806381 3027 webserver.cc:460] Webserver started at http://127.0.198.195:37309/ using document root <none> and password file <none>
I20260807 19:43:49.807281 3027 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:49.807490 3027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:49.807821 3027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:49.811298 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "04d83bc9bfc540e399d2e94b5198d526"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "3924581c144b785d017d713592c6bbfc"
server_key_iv: "6d846d91a55bfeb0d6ad6d6f8a25f3b0"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.812302 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "04d83bc9bfc540e399d2e94b5198d526"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "3924581c144b785d017d713592c6bbfc"
server_key_iv: "6d846d91a55bfeb0d6ad6d6f8a25f3b0"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.818809 3027 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:43:49.823529 3044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:49.825487 3027 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:43:49.825738 3027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "04d83bc9bfc540e399d2e94b5198d526"
format_stamp: "Formatted at 2026-08-07 19:43:49 on dist-test-slave-btw8"
server_key: "3924581c144b785d017d713592c6bbfc"
server_key_iv: "6d846d91a55bfeb0d6ad6d6f8a25f3b0"
server_key_version: "encryptionkey@0"
I20260807 19:43:49.826027 3027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:49.864482 3027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:49.869971 3027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:49.870317 3027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:49.873425 3027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:49.873588 3027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:49.873750 3027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:49.873854 3027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:43:49 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:49.907238 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.875059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.195:36275)
0807 19:43:49.876007 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:49.876060 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:49.878302 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:49.879835 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:49.879848 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:49.880920 (+ 1072us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:49.882024 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.882053 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.884149 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.884155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.887160 (+ 3005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.887204 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.888926 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.888939 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.889401 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.890448 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:49.890496 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:49.895043 (+ 4547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:49.899934 (+ 4891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.899966 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.899995 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.901137 (+ 1142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.901846 (+ 709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.901869 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.901876 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.902304 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.903342 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:49.903383 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:49.904116 (+ 733us) client_negotiation.cc:769] Sending connection context
0807 19:43:49.904553 (+ 437us) client_negotiation.cc:240] Negotiation successful
0807 19:43:49.905246 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":437,"threads_started":1}
I20260807 19:43:49.905980 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.874339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:36275 (local address 127.0.198.254:33337)
0807 19:43:49.874884 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:49.874893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:49.876231 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 19:43:49.878741 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:49.878751 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:49.878946 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:49.879055 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:49.882407 (+ 3352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.883904 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.887537 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.888548 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.888787 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.895563 (+ 6776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:49.895571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:49.895574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:49.895618 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:49.899554 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.901378 (+ 1824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.901385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.901389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.901679 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.902504 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.902511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.902517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.903058 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:49.903250 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:49.904822 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0807 19:43:49.905263 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":272,"threads_started":1}
I20260807 19:43:49.943818 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.916459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:40535 (local address 127.0.198.254:33337)
0807 19:43:49.923695 (+ 7236us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:49.923706 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:49.924586 (+ 880us) server_negotiation.cc:407] Connection header received
0807 19:43:49.924844 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:49.924854 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:49.924983 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:49.925113 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:49.927315 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.928733 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.931464 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.932165 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.932346 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.935921 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:49.935934 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:49.935941 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:49.935996 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:49.940066 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.941313 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.941321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.941325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.941507 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:49.941975 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:49.941981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:49.941985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:49.942358 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:49.942588 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:49.943055 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 19:43:49.943389 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7008}
I20260807 19:43:49.944119 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:49.916895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.198.195:40535)
0807 19:43:49.924410 (+ 7515us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:49.924444 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:49.924670 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:49.925387 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:49.925391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:49.926241 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:49.927038 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.927061 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.929034 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.929042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.931168 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:49.931192 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:49.932438 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:49.932446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:49.932677 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:49.934209 (+ 1532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:49.934247 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:49.935464 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:49.940340 (+ 4876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.940345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.940349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.941118 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.941661 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:49.941666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:49.941675 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:49.941826 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:49.942506 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:49.942514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:49.942754 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 19:43:49.942921 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 19:43:49.943402 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7333,"thread_start_us":6643,"threads_started":1}
I20260807 19:43:49.955643 3027 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:32963
I20260807 19:43:49.955770 3170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:32963 every 8 connection(s)
I20260807 19:43:49.957056 3027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:49.959872 3171 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33337
I20260807 19:43:49.960189 3171 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:49.960980 3171 heartbeater.cc:511] Master 127.0.198.254:33337 requested a full tablet report, sending...
I20260807 19:43:49.962771 2655 ts_manager.cc:194] Registered new tserver with Master: 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:49.963909 2655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:36275
I20260807 19:43:49.964756 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3027
I20260807 19:43:49.964944 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:49.965482 795 external_mini_cluster.cc:1428] Setting key 130e72363e6152772b575b1fb8ec91d6
I20260807 19:43:49.969090 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:43:50 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:43:50 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:50.037127 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.018579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.0.198.254:33337)
0807 19:43:50.018923 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.018932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.019345 (+ 413us) server_negotiation.cc:407] Connection header received
0807 19:43:50.019696 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.019701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.019783 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.019916 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:50.021665 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.022725 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.024336 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.024798 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.024963 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.029873 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:50.029881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:50.029885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:50.029950 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:50.033726 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:50.034661 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:50.034668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:50.034671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:50.034829 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:50.035325 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:50.035330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:50.035332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:50.035749 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:50.036082 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.036437 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.036710 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 19:43:50.042802 2655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:50.051626 2655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:50.086949 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.066942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.0.198.195:32963)
0807 19:43:50.077419 (+ 10477us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.077427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.077467 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:43:50.077590 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.077598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.077811 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.078532 (+ 721us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.078925 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.080894 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.083718 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085646 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085809 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.086058 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.086373 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.086558 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10278,"thread_start_us":10054,"threads_started":1}
I20260807 19:43:50.086961 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.071072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.0.198.193:38953)
0807 19:43:50.071984 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.071999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.077071 (+ 5072us) server_negotiation.cc:407] Connection header received
0807 19:43:50.077488 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.077494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.077756 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.077992 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.079566 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.081102 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.083690 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085280 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085398 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.085600 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.086413 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.086614 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":456,"threads_started":1}
I20260807 19:43:50.086969 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.067235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.0.198.194:38957)
0807 19:43:50.076072 (+ 8837us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.076086 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.076748 (+ 662us) server_negotiation.cc:407] Connection header received
0807 19:43:50.077012 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.077038 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.077343 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.077610 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.078485 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.080186 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.083335 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085050 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085259 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.085525 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.086478 (+ 953us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.086649 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8599,"thread_start_us":8454,"threads_started":1}
I20260807 19:43:50.086993 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.066559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:32963 (local address 127.0.0.1:54674)
0807 19:43:50.068290 (+ 1731us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.068349 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.068605 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.078085 (+ 9480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.078093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.078126 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.078727 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.078744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.081057 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.081063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.083544 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.083558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085946 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.086140 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.086177 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.086322 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.086558 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":74,"thread_start_us":1039,"threads_started":1}
I20260807 19:43:50.087879 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.067242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.0.1:45590)
0807 19:43:50.076534 (+ 9292us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.076559 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.076747 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.077622 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.077631 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.077665 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.078276 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.078292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.080401 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.080406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.083110 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.083129 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085700 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085710 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.086026 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.086156 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.086311 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.087324 (+ 1013us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9066,"mutex_wait_us":882,"thread_start_us":8902,"threads_started":1}
I20260807 19:43:50.087903 3189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.070263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:38953 (local address 127.0.0.1:60120)
0807 19:43:50.076901 (+ 6638us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.076927 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.077210 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.078013 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.078027 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.078062 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.079278 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.079297 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.081268 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.081279 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.083281 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.083301 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.085872 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.085880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.086105 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.086152 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.087277 (+ 1125us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.087428 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6467,"mutex_wait_us":105,"thread_start_us":6240,"threads_started":1}
I20260807 19:43:50.091849 2802 tablet_service.cc:1467] Processing CreateTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 (DEFAULT_TABLE table=test-table [id=ed75f7c2d38a4857bd9375a43a784f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.091849 3095 tablet_service.cc:1467] Processing CreateTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 (DEFAULT_TABLE table=test-table [id=ed75f7c2d38a4857bd9375a43a784f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.093498 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd17fbf92efd41e99bc0a13d540a99b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.093488 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd17fbf92efd41e99bc0a13d540a99b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.093150 2950 tablet_service.cc:1467] Processing CreateTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 (DEFAULT_TABLE table=test-table [id=ed75f7c2d38a4857bd9375a43a784f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.094123 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd17fbf92efd41e99bc0a13d540a99b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.106020 3191 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Bootstrap starting.
I20260807 19:43:50.110078 3193 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Bootstrap starting.
I20260807 19:43:50.110559 3191 tablet_bootstrap.cc:654] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.112239 3192 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Bootstrap starting.
I20260807 19:43:50.113072 3191 log.cc:824] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:50.114570 3193 tablet_bootstrap.cc:654] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.116154 3193 log.cc:824] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:50.116462 3192 tablet_bootstrap.cc:654] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.117234 3191 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: No bootstrap required, opened a new log
I20260807 19:43:50.117599 3191 ts_tablet_manager.cc:1397] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 19:43:50.118170 3192 log.cc:824] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:50.120383 3193 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: No bootstrap required, opened a new log
I20260807 19:43:50.120721 3193 ts_tablet_manager.cc:1397] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 19:43:50.126606 3192 tablet_bootstrap.cc:492] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: No bootstrap required, opened a new log
I20260807 19:43:50.127008 3192 ts_tablet_manager.cc:1397] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260807 19:43:50.131225 3191 raft_consensus.cc:348] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.131981 3191 raft_consensus.cc:374] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.132107 3191 raft_consensus.cc:729] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.133616 3191 consensus_queue.cc:260] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.133507 3193 raft_consensus.cc:348] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.134029 3193 raft_consensus.cc:374] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.134166 3193 raft_consensus.cc:729] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.135041 3022 heartbeater.cc:503] Master 127.0.198.254:33337 was elected leader, sending a full tablet report...
I20260807 19:43:50.135003 3193 consensus_queue.cc:260] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.136034 3191 ts_tablet_manager.cc:1428] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
I20260807 19:43:50.137277 3171 heartbeater.cc:503] Master 127.0.198.254:33337 was elected leader, sending a full tablet report...
I20260807 19:43:50.137547 3192 raft_consensus.cc:348] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.137970 3192 raft_consensus.cc:374] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.138096 3192 raft_consensus.cc:729] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.138908 3192 consensus_queue.cc:260] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.139189 3193 ts_tablet_manager.cc:1428] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260807 19:43:50.141237 3192 ts_tablet_manager.cc:1428] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260807 19:43:50.209069 3172 tablet.cc:2369] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:43:50.238605 2875 tablet.cc:2369] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:50.289763 3200 raft_consensus.cc:482] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:50.290086 3200 raft_consensus.cc:504] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.291882 3200 leader_election.cc:288] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963), 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957)
I20260807 19:43:50.299427 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.292428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:34639 (local address 127.0.198.195:32963)
0807 19:43:50.292676 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.292684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.293287 (+ 603us) server_negotiation.cc:407] Connection header received
0807 19:43:50.293524 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.293533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.293615 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.293711 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.294367 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.295158 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.297220 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.298285 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.298428 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.298488 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.298920 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.299072 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:43:50.299510 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.292278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:32963 (local address 127.0.198.193:34639)
0807 19:43:50.293136 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.293164 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.293338 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.293767 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.293773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.293803 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.294209 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.294223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.295304 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.295310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.297027 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.297043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.298516 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.298523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.298717 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.298758 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.298856 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.299028 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":414,"threads_started":1}
I20260807 19:43:50.299865 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.292492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:44097 (local address 127.0.198.194:38957)
0807 19:43:50.292731 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.292737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.293422 (+ 685us) server_negotiation.cc:407] Connection header received
0807 19:43:50.293601 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.293606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.293712 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.293810 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.294580 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.295535 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.297778 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.298719 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.298903 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.298977 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.299416 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.299594 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 19:43:50.300568 3202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.292413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.198.193:44097)
0807 19:43:50.293273 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.293292 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.293458 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.293859 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.293864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.293926 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.294344 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.294358 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.295744 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.295752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.297592 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.297609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.299000 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.299007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.299197 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.299236 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.299383 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.300113 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"spinlock_wait_cycles":3584,"thread_start_us":499,"threads_started":1}
I20260807 19:43:50.300863 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd17fbf92efd41e99bc0a13d540a99b3" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526" is_pre_election: true
I20260807 19:43:50.301467 3125 raft_consensus.cc:2454] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:50.301698 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd17fbf92efd41e99bc0a13d540a99b3" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30" is_pre_election: true
I20260807 19:43:50.302294 2972 raft_consensus.cc:2454] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:50.302349 2751 leader_election.cc:302] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:50.302874 3200 raft_consensus.cc:2781] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:50.303030 3200 raft_consensus.cc:482] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:50.303172 3200 raft_consensus.cc:3037] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.305742 3200 raft_consensus.cc:504] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
I20260807 19:43:50.306586 3200 leader_election.cc:288] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Requested vote from peers 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963), 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957)
I20260807 19:43:50.307191 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd17fbf92efd41e99bc0a13d540a99b3" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526"
I20260807 19:43:50.307302 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd17fbf92efd41e99bc0a13d540a99b3" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30"
I20260807 19:43:50.307523 3125 raft_consensus.cc:3037] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.307606 2972 raft_consensus.cc:3037] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.310714 3125 raft_consensus.cc:2454] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:50.310760 2972 raft_consensus.cc:2454] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:50.311314 2751 leader_election.cc:302] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:50.311769 3200 raft_consensus.cc:2781] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:50.312361 3200 raft_consensus.cc:686] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Becoming Leader. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Running, Role: LEADER
I20260807 19:43:50.313015 3200 consensus_queue.cc:237] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } }
W20260807 19:43:50.319697 3024 tablet.cc:2369] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:50.319172 2654 catalog_manager.cc:5495] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0511bae4536d4286bb89b2552e693b16 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "0511bae4536d4286bb89b2552e693b16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } health_report { overall_health: HEALTHY } } }
Aug 07 19:43:50 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:50.429917 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.410330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33337 (local address 127.0.0.1:35016)
0807 19:43:50.411164 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.411233 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.411516 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.412034 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.412038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.412987 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:50.413279 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.413290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.415200 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.415205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.416502 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.416519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.417847 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.417853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.417981 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.418898 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:50.418925 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:50.421563 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:50.426736 (+ 5173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:50.426765 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:50.426772 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:50.427327 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:50.427936 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:50.427941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:50.427943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:50.428135 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:50.428972 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:50.428979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:50.429174 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.429324 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.429570 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":464,"threads_started":1}
I20260807 19:43:50.430392 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.410290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.0.198.254:33337)
0807 19:43:50.410581 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.410590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.411381 (+ 791us) server_negotiation.cc:407] Connection header received
0807 19:43:50.411726 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.411732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.411829 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.411972 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:50.413469 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.414973 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.416686 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.417594 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.417777 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.421839 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:50.421850 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:50.421858 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:50.421916 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:50.426464 (+ 4548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:50.427561 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:50.427570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:50.427575 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:50.427768 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:50.428284 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:50.428291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:50.428297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:50.428795 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:50.428996 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.429505 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.429829 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:43:50.436395 2654 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:35016:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:43:50.437613 2654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:50.447129 3095 tablet_service.cc:1467] Processing CreateTablet for tablet ab16486d75244037bf002aa661d8e73a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bab54835d414cf8bfbcc2d2841582fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:50.446864 2802 tablet_service.cc:1467] Processing CreateTablet for tablet ab16486d75244037bf002aa661d8e73a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bab54835d414cf8bfbcc2d2841582fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:50.448062 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab16486d75244037bf002aa661d8e73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.448062 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab16486d75244037bf002aa661d8e73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.447268 2950 tablet_service.cc:1467] Processing CreateTablet for tablet ab16486d75244037bf002aa661d8e73a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bab54835d414cf8bfbcc2d2841582fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:43:50.448573 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab16486d75244037bf002aa661d8e73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.455984 3193 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: Bootstrap starting.
I20260807 19:43:50.457955 3193 tablet_bootstrap.cc:654] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.458492 3192 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: Bootstrap starting.
I20260807 19:43:50.459520 3191 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: Bootstrap starting.
I20260807 19:43:50.460671 3192 tablet_bootstrap.cc:654] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.461634 3191 tablet_bootstrap.cc:654] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.464808 3192 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: No bootstrap required, opened a new log
I20260807 19:43:50.465049 3192 ts_tablet_manager.cc:1397] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 19:43:50.465615 3191 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: No bootstrap required, opened a new log
I20260807 19:43:50.465812 3191 ts_tablet_manager.cc:1397] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260807 19:43:50.466467 3192 raft_consensus.cc:348] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.466753 3193 tablet_bootstrap.cc:492] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: No bootstrap required, opened a new log
I20260807 19:43:50.466850 3192 raft_consensus.cc:374] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.466727 3191 raft_consensus.cc:348] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.466974 3193 ts_tablet_manager.cc:1397] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20260807 19:43:50.466989 3192 raft_consensus.cc:729] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.467031 3191 raft_consensus.cc:374] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.467231 3191 raft_consensus.cc:729] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.467729 3191 consensus_queue.cc:260] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.468115 3193 raft_consensus.cc:348] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.468449 3193 raft_consensus.cc:374] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.468576 3193 raft_consensus.cc:729] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.468277 3198 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.468722 3198 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.468888 3193 consensus_queue.cc:260] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.469592 3193 ts_tablet_manager.cc:1428] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 19:43:50.469470 3192 consensus_queue.cc:260] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.469492 3199 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.469894 3199 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.470311 3191 ts_tablet_manager.cc:1428] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 19:43:50.470189 3200 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.470554 3200 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.472465 3192 ts_tablet_manager.cc:1428] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260807 19:43:50.558552 3198 raft_consensus.cc:482] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:50.558873 3198 raft_consensus.cc:504] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.560732 3198 leader_election.cc:288] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:50.570652 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.562437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:35229 (local address 127.0.198.195:32963)
0807 19:43:50.562640 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.562648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.563295 (+ 647us) server_negotiation.cc:407] Connection header received
0807 19:43:50.563551 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.563556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.563619 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.563720 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.564318 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.565145 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.568322 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.569491 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.569692 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.569790 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.570177 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.570326 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:43:50.571630 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.561375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:54437 (local address 127.0.198.193:38953)
0807 19:43:50.561694 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.561703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.562149 (+ 446us) server_negotiation.cc:407] Connection header received
0807 19:43:50.562422 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.562428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.562526 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.562596 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:50.563845 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.565178 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.568315 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.570087 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.570305 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.570412 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.571034 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.571249 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:43:50.571962 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.562267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:32963 (local address 127.0.198.194:35229)
0807 19:43:50.563178 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.563205 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.563388 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.563738 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.563742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.563766 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.564132 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.564141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.565302 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.565310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.568093 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.568109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.569757 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.569764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.569961 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.569997 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.570948 (+ 951us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.571132 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":62,"thread_start_us":525,"threads_started":1}
I20260807 19:43:50.572718 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.561297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:38953 (local address 127.0.198.194:54437)
0807 19:43:50.562016 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.562033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.562229 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.562869 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.562875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.562912 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:50.563596 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.563620 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.565359 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.565365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.568089 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.568113 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.570600 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.570608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.570793 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.570825 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.571862 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.572076 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":88,"thread_start_us":423,"threads_started":1}
I20260807 19:43:50.572712 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab16486d75244037bf002aa661d8e73a" candidate_uuid: "3c38bb9708a04807881f1ac58062ba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526" is_pre_election: true
I20260807 19:43:50.573050 3125 raft_consensus.cc:2454] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c38bb9708a04807881f1ac58062ba30 in term 0.
I20260807 19:43:50.573732 2895 leader_election.cc:302] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 3c38bb9708a04807881f1ac58062ba30; no voters:
I20260807 19:43:50.574007 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab16486d75244037bf002aa661d8e73a" candidate_uuid: "3c38bb9708a04807881f1ac58062ba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511bae4536d4286bb89b2552e693b16" is_pre_election: true
I20260807 19:43:50.574307 3198 raft_consensus.cc:2781] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:50.574466 3198 raft_consensus.cc:482] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:50.574587 3198 raft_consensus.cc:3037] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.574587 2823 raft_consensus.cc:2454] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c38bb9708a04807881f1ac58062ba30 in term 0.
I20260807 19:43:50.578831 3198 raft_consensus.cc:504] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.579815 3198 leader_election.cc:288] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [CANDIDATE]: Term 1 election: Requested vote from peers 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:50.580694 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab16486d75244037bf002aa661d8e73a" candidate_uuid: "3c38bb9708a04807881f1ac58062ba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526"
I20260807 19:43:50.581027 3125 raft_consensus.cc:3037] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.583492 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab16486d75244037bf002aa661d8e73a" candidate_uuid: "3c38bb9708a04807881f1ac58062ba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511bae4536d4286bb89b2552e693b16"
I20260807 19:43:50.583798 2823 raft_consensus.cc:3037] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.588114 2823 raft_consensus.cc:2454] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c38bb9708a04807881f1ac58062ba30 in term 1.
I20260807 19:43:50.588901 2895 leader_election.cc:302] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0511bae4536d4286bb89b2552e693b16, 3c38bb9708a04807881f1ac58062ba30; no voters:
I20260807 19:43:50.589290 3125 raft_consensus.cc:2454] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c38bb9708a04807881f1ac58062ba30 in term 1.
I20260807 19:43:50.589332 3198 raft_consensus.cc:2781] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:50.590263 3198 raft_consensus.cc:686] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [term 1 LEADER]: Becoming Leader. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Running, Role: LEADER
I20260807 19:43:50.590868 3198 consensus_queue.cc:237] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.595408 3216 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: TxnStatusTablet state changed. Reason: New leader 3c38bb9708a04807881f1ac58062ba30. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.596002 3216 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:50.595999 2655 catalog_manager.cc:5495] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } health_report { overall_health: UNKNOWN } } }
I20260807 19:43:50.597584 3219 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:43:50.597899 3219 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:43:50.641319 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.634086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.0.1:45592)
0807 19:43:50.634466 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.634484 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.634607 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.635240 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.635245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.635259 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:50.635518 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.635529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.636927 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.636934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.638146 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.638164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.639135 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.639144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.639451 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.639536 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.640603 (+ 1067us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.640648 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.640777 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.640990 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260807 19:43:50.641316 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.634108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.0.198.194:38957)
0807 19:43:50.634856 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.634861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.634877 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:43:50.634956 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.634960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.635043 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.635209 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:50.635678 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.636740 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.638355 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.638910 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.639090 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.639788 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.640454 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.640650 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.640804 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.640910 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626}
I20260807 19:43:50.651327 2823 raft_consensus.cc:1261] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Refusing update from remote peer 3c38bb9708a04807881f1ac58062ba30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:50.651515 3125 raft_consensus.cc:1261] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Refusing update from remote peer 3c38bb9708a04807881f1ac58062ba30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:50.652971 3216 consensus_queue.cc:1035] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:50.653486 3198 consensus_queue.cc:1035] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:50.662846 3199 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: TxnStatusTablet state changed. Reason: New leader 3c38bb9708a04807881f1ac58062ba30. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.663178 3199 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.667687 3200 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: TxnStatusTablet state changed. Reason: New leader 3c38bb9708a04807881f1ac58062ba30. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.668093 3200 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.669198 3222 mvcc.cc:205] Tried to move back new op lower bound from 7315995978335576064 to 7315995978104516608. Current Snapshot: MvccSnapshot[applied={T|T < 7315995978335576064}]
I20260807 19:43:50.670569 3198 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.670984 3198 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:50.680752 3216 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.681092 3216 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 3c38bb9708a04807881f1ac58062ba30: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:43:50.684643 3199 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.684963 3199 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 04d83bc9bfc540e399d2e94b5198d526: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.693272 3200 tablet_replica.cc:439] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c38bb9708a04807881f1ac58062ba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } } }
I20260807 19:43:50.693614 3200 tablet_replica.cc:442] T ab16486d75244037bf002aa661d8e73a P 0511bae4536d4286bb89b2552e693b16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:43:50.702639 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.692808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.0.198.193:38953)
0807 19:43:50.692990 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.692997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.696031 (+ 3034us) server_negotiation.cc:407] Connection header received
0807 19:43:50.696359 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.696365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.696462 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.696589 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:50.697141 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.697963 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.700115 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.700626 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.700797 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.701345 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.701938 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.702060 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.702221 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.702297 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:43:50.723073 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.715830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40737 (local address 127.0.198.194:38957)
0807 19:43:50.716081 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.716089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.716773 (+ 684us) server_negotiation.cc:407] Connection header received
0807 19:43:50.717018 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.717023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.717113 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.717209 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:50.717816 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.718607 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.720858 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.721366 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.721492 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.722237 (+ 745us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.722365 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.722453 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.722681 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.722818 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:43:50.723425 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.715839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.198.193:40737)
0807 19:43:50.716640 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.716663 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.716849 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.717257 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.717263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.717277 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:50.717643 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.717656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.718762 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.718769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.720656 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.720677 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.721597 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.721605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.721804 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.721897 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.722487 (+ 590us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.722503 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.722597 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.722774 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":416,"threads_started":1}
I20260807 19:43:50.735423 3125 raft_consensus.cc:1261] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:50.735416 2972 raft_consensus.cc:1261] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:50.736265 3200 consensus_queue.cc:1035] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:50.736967 3203 consensus_queue.cc:1035] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:50.760891 3225 mvcc.cc:205] Tried to move back new op lower bound from 7315995978684272640 to 7315995976967766016. Current Snapshot: MvccSnapshot[applied={T|T < 7315995978684272640}]
I20260807 19:43:50.765188 3224 mvcc.cc:205] Tried to move back new op lower bound from 7315995978684272640 to 7315995976967766016. Current Snapshot: MvccSnapshot[applied={T|T < 7315995978684272640}]
I20260807 19:43:50.799078 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.791370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:56217 (local address 127.0.198.193:38953)
0807 19:43:50.791595 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:50.791602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:50.792401 (+ 799us) server_negotiation.cc:407] Connection header received
0807 19:43:50.792701 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:50.792706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:50.792798 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:50.792890 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:50.793743 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.794455 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.796803 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.797500 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.797607 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.798318 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.798434 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.798529 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:50.798712 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:43:50.798789 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:43:50.799506 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:50.791417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:38953 (local address 127.0.198.194:56217)
0807 19:43:50.792260 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:50.792302 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:50.792464 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:50.793080 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:50.793086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:50.793100 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:50.793514 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.793528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.794589 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.794595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.796625 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:50.796641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:50.797821 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:50.797828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:50.798017 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:50.798099 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:50.798563 (+ 464us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:50.798575 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:43:50.798667 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:43:50.798826 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":453,"threads_started":1}
I20260807 19:43:50.850054 2654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:50.850386 2654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:50.855115 2654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Sending DeleteTablet for 3 replicas of tablet fd17fbf92efd41e99bc0a13d540a99b3
I20260807 19:43:50.856563 3095 tablet_service.cc:1514] Processing DeleteTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:50 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:54674
I20260807 19:43:50.856570 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:50 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45590
I20260807 19:43:50.858049 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet fd17fbf92efd41e99bc0a13d540a99b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:50 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60120
I20260807 19:43:50.863505 2654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:50.864593 2654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:50.874277 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 95fafcca345e470ca71b029ea40f6cef (DEFAULT_TABLE table=test-table [id=010d327000ff456ebc2becf3146de38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.874986 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fafcca345e470ca71b029ea40f6cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.876523 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 95fafcca345e470ca71b029ea40f6cef (DEFAULT_TABLE table=test-table [id=010d327000ff456ebc2becf3146de38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.877133 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fafcca345e470ca71b029ea40f6cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.880489 3094 tablet_service.cc:1467] Processing CreateTablet for tablet 95fafcca345e470ca71b029ea40f6cef (DEFAULT_TABLE table=test-table [id=010d327000ff456ebc2becf3146de38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:50.881102 3094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fafcca345e470ca71b029ea40f6cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:50.884656 3192 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Bootstrap starting.
I20260807 19:43:50.885274 3191 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Bootstrap starting.
I20260807 19:43:50.886729 3192 tablet_bootstrap.cc:654] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.886981 3191 tablet_bootstrap.cc:654] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.889420 3193 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Bootstrap starting.
I20260807 19:43:50.892006 3193 tablet_bootstrap.cc:654] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:50.895241 3247 tablet_replica.cc:330] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: stopping tablet replica
I20260807 19:43:50.895859 3247 raft_consensus.cc:2229] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:50.896740 3247 raft_consensus.cc:2258] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:50.898031 3191 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: No bootstrap required, opened a new log
I20260807 19:43:50.898253 3191 ts_tablet_manager.cc:1397] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.002s
I20260807 19:43:50.899223 3191 raft_consensus.cc:348] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.899680 3191 raft_consensus.cc:374] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.899785 3191 raft_consensus.cc:729] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.900151 3191 consensus_queue.cc:260] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.901216 3191 ts_tablet_manager.cc:1428] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 19:43:50.902926 3244 tablet_replica.cc:330] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: stopping tablet replica
I20260807 19:43:50.903231 3193 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: No bootstrap required, opened a new log
I20260807 19:43:50.903465 3193 ts_tablet_manager.cc:1397] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260807 19:43:50.903481 3244 raft_consensus.cc:2229] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:50.903862 3244 raft_consensus.cc:2258] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:50.904471 3193 raft_consensus.cc:348] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.904775 3192 tablet_bootstrap.cc:492] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: No bootstrap required, opened a new log
I20260807 19:43:50.904784 3193 raft_consensus.cc:374] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.904879 3193 raft_consensus.cc:729] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.904973 3192 ts_tablet_manager.cc:1397] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.004s
I20260807 19:43:50.905220 3193 consensus_queue.cc:260] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.905786 3192 raft_consensus.cc:348] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.906114 3192 raft_consensus.cc:374] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:50.906220 3192 raft_consensus.cc:729] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Initialized, Role: FOLLOWER
I20260807 19:43:50.906328 3247 ts_tablet_manager.cc:1905] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:50.906476 3192 consensus_queue.cc:260] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.907560 3193 ts_tablet_manager.cc:1428] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:43:50.908114 3192 ts_tablet_manager.cc:1428] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 19:43:50.913868 3244 ts_tablet_manager.cc:1905] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:50.918939 3246 tablet_replica.cc:330] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: stopping tablet replica
I20260807 19:43:50.919652 3246 raft_consensus.cc:2229] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:50.920207 3246 raft_consensus.cc:2258] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:50.922950 3244 ts_tablet_manager.cc:1918] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:50.923177 3244 log.cc:1196] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/fd17fbf92efd41e99bc0a13d540a99b3
I20260807 19:43:50.924500 3246 ts_tablet_manager.cc:1905] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:50.928050 3247 ts_tablet_manager.cc:1918] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:50.928313 3247 log.cc:1196] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/fd17fbf92efd41e99bc0a13d540a99b3
I20260807 19:43:50.928949 3247 ts_tablet_manager.cc:1939] T fd17fbf92efd41e99bc0a13d540a99b3 P 0511bae4536d4286bb89b2552e693b16: Deleting consensus metadata
I20260807 19:43:50.931324 3244 ts_tablet_manager.cc:1939] T fd17fbf92efd41e99bc0a13d540a99b3 P 3c38bb9708a04807881f1ac58062ba30: Deleting consensus metadata
I20260807 19:43:50.932404 2640 catalog_manager.cc:4841] TS 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953): tablet fd17fbf92efd41e99bc0a13d540a99b3 (table test-table [id=ed75f7c2d38a4857bd9375a43a784f81]) successfully deleted
I20260807 19:43:50.934671 2641 catalog_manager.cc:4841] TS 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957): tablet fd17fbf92efd41e99bc0a13d540a99b3 (table test-table [id=ed75f7c2d38a4857bd9375a43a784f81]) successfully deleted
I20260807 19:43:50.936677 3246 ts_tablet_manager.cc:1918] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:50.936882 3246 log.cc:1196] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/fd17fbf92efd41e99bc0a13d540a99b3
I20260807 19:43:50.937515 3246 ts_tablet_manager.cc:1939] T fd17fbf92efd41e99bc0a13d540a99b3 P 04d83bc9bfc540e399d2e94b5198d526: Deleting consensus metadata
I20260807 19:43:50.939946 2640 catalog_manager.cc:4841] TS 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963): tablet fd17fbf92efd41e99bc0a13d540a99b3 (table test-table [id=ed75f7c2d38a4857bd9375a43a784f81]) successfully deleted
I20260807 19:43:50.946692 3238 raft_consensus.cc:482] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:50.946940 3238 raft_consensus.cc:504] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.947911 3238 leader_election.cc:288] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:50.948740 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fafcca345e470ca71b029ea40f6cef" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526" is_pre_election: true
I20260807 19:43:50.949015 3125 raft_consensus.cc:2454] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:50.949012 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fafcca345e470ca71b029ea40f6cef" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30" is_pre_election: true
I20260807 19:43:50.949354 2972 raft_consensus.cc:2454] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:50.949488 2751 leader_election.cc:302] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:50.949829 3238 raft_consensus.cc:2781] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:50.949955 3238 raft_consensus.cc:482] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:50.950062 3238 raft_consensus.cc:3037] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.953732 3238 raft_consensus.cc:504] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.955209 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fafcca345e470ca71b029ea40f6cef" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30"
I20260807 19:43:50.955510 2972 raft_consensus.cc:3037] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.955435 3125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fafcca345e470ca71b029ea40f6cef" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526"
I20260807 19:43:50.955696 3125 raft_consensus.cc:3037] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:50.959223 2972 raft_consensus.cc:2454] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:50.959343 3125 raft_consensus.cc:2454] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:50.960000 2751 leader_election.cc:302] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:50.962067 3200 raft_consensus.cc:2781] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:50.962327 3200 raft_consensus.cc:686] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Becoming Leader. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Running, Role: LEADER
I20260807 19:43:50.963110 3200 consensus_queue.cc:237] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:50.963547 3238 leader_election.cc:288] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Requested vote from peers 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:50.975927 2654 catalog_manager.cc:5495] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0511bae4536d4286bb89b2552e693b16 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "0511bae4536d4286bb89b2552e693b16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } health_report { overall_health: UNKNOWN } } }
I20260807 19:43:51.024641 2972 raft_consensus.cc:1261] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:51.025558 3124 raft_consensus.cc:1261] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:51.026711 3238 consensus_queue.cc:1035] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:51.027601 3200 consensus_queue.cc:1035] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:43:51.079137 3225 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:43:52.690416 2654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:52.690843 2654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:52.696760 2654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Sending DeleteTablet for 3 replicas of tablet 95fafcca345e470ca71b029ea40f6cef
I20260807 19:43:52.697942 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fafcca345e470ca71b029ea40f6cef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:52 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45590
I20260807 19:43:52.698220 3095 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fafcca345e470ca71b029ea40f6cef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:52 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:54674
I20260807 19:43:52.698496 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fafcca345e470ca71b029ea40f6cef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:52 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60120
I20260807 19:43:52.698688 3277 tablet_replica.cc:330] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: stopping tablet replica
I20260807 19:43:52.698961 3277 raft_consensus.cc:2229] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:52.699293 3277 raft_consensus.cc:2258] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:52.700815 3277 ts_tablet_manager.cc:1905] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:52.711016 3279 tablet_replica.cc:330] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: stopping tablet replica
I20260807 19:43:52.711110 3277 ts_tablet_manager.cc:1918] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:52.711390 3279 raft_consensus.cc:2229] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:52.711472 3277 log.cc:1196] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/95fafcca345e470ca71b029ea40f6cef
I20260807 19:43:52.712080 3279 raft_consensus.cc:2258] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:52.712244 3277 ts_tablet_manager.cc:1939] T 95fafcca345e470ca71b029ea40f6cef P 3c38bb9708a04807881f1ac58062ba30: Deleting consensus metadata
I20260807 19:43:52.713454 3279 ts_tablet_manager.cc:1905] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:52.714036 2641 catalog_manager.cc:4841] TS 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957): tablet 95fafcca345e470ca71b029ea40f6cef (table test-table [id=010d327000ff456ebc2becf3146de38b]) successfully deleted
I20260807 19:43:52.715529 3278 tablet_replica.cc:330] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: stopping tablet replica
I20260807 19:43:52.715756 3278 raft_consensus.cc:2229] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:52.716030 3278 raft_consensus.cc:2258] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:52.717280 3278 ts_tablet_manager.cc:1905] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:52.721944 3279 ts_tablet_manager.cc:1918] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:52.722191 3279 log.cc:1196] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/95fafcca345e470ca71b029ea40f6cef
I20260807 19:43:52.722734 3279 ts_tablet_manager.cc:1939] T 95fafcca345e470ca71b029ea40f6cef P 0511bae4536d4286bb89b2552e693b16: Deleting consensus metadata
I20260807 19:43:52.724553 2640 catalog_manager.cc:4841] TS 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953): tablet 95fafcca345e470ca71b029ea40f6cef (table test-table [id=010d327000ff456ebc2becf3146de38b]) successfully deleted
I20260807 19:43:52.726011 3278 ts_tablet_manager.cc:1918] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:52.726174 3278 log.cc:1196] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/95fafcca345e470ca71b029ea40f6cef
I20260807 19:43:52.726926 3278 ts_tablet_manager.cc:1939] T 95fafcca345e470ca71b029ea40f6cef P 04d83bc9bfc540e399d2e94b5198d526: Deleting consensus metadata
I20260807 19:43:52.728509 2640 catalog_manager.cc:4841] TS 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963): tablet 95fafcca345e470ca71b029ea40f6cef (table test-table [id=010d327000ff456ebc2becf3146de38b]) successfully deleted
Aug 07 19:43:52 dist-test-slave-btw8 krb5kdc[2611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:43:52 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:52.779524 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:52.760153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.0.198.254:33337)
0807 19:43:52.761135 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:52.761140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:52.761163 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:43:52.761221 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:52.761224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:52.761302 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:52.761407 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:52.763531 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:52.764590 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:52.766819 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:52.767517 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:52.767634 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:52.772521 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:52.772528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:52.772531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:52.772577 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:52.776096 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:52.777081 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:52.777088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:52.777092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:52.777261 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:52.777777 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:52.777784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:52.777787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:52.778107 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:52.778398 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:52.778746 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:43:52.779011 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":691,"threads_started":1}
I20260807 19:43:52.784236 2654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:52.784965 2654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:52.792964 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 534a2e7feb08446e8456dfa658835911 (DEFAULT_TABLE table=test-table [id=b50d09f8615b4cfb8970eb740920a6c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:52.793015 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 534a2e7feb08446e8456dfa658835911 (DEFAULT_TABLE table=test-table [id=b50d09f8615b4cfb8970eb740920a6c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:52.793354 3095 tablet_service.cc:1467] Processing CreateTablet for tablet 534a2e7feb08446e8456dfa658835911 (DEFAULT_TABLE table=test-table [id=b50d09f8615b4cfb8970eb740920a6c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:52.793490 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 534a2e7feb08446e8456dfa658835911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:52.793541 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 534a2e7feb08446e8456dfa658835911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:52.793808 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 534a2e7feb08446e8456dfa658835911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:52.800902 3288 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Bootstrap starting.
I20260807 19:43:52.801642 3289 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Bootstrap starting.
I20260807 19:43:52.802275 3290 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Bootstrap starting.
I20260807 19:43:52.803613 3288 tablet_bootstrap.cc:654] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:52.803613 3289 tablet_bootstrap.cc:654] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:52.803995 3290 tablet_bootstrap.cc:654] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:52.806161 3289 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: No bootstrap required, opened a new log
I20260807 19:43:52.806305 3288 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: No bootstrap required, opened a new log
I20260807 19:43:52.806339 3289 ts_tablet_manager.cc:1397] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 19:43:52.806450 3290 tablet_bootstrap.cc:492] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: No bootstrap required, opened a new log
I20260807 19:43:52.806485 3288 ts_tablet_manager.cc:1397] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 19:43:52.806632 3290 ts_tablet_manager.cc:1397] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260807 19:43:52.807266 3289 raft_consensus.cc:348] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.807446 3290 raft_consensus.cc:348] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.807430 3288 raft_consensus.cc:348] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.807651 3289 raft_consensus.cc:374] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:52.807693 3290 raft_consensus.cc:374] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:52.807706 3288 raft_consensus.cc:374] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:52.807756 3289 raft_consensus.cc:729] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Initialized, Role: FOLLOWER
I20260807 19:43:52.807790 3290 raft_consensus.cc:729] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Initialized, Role: FOLLOWER
I20260807 19:43:52.807807 3288 raft_consensus.cc:729] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Initialized, Role: FOLLOWER
I20260807 19:43:52.808017 3289 consensus_queue.cc:260] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.808037 3288 consensus_queue.cc:260] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.808624 3288 ts_tablet_manager.cc:1428] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 19:43:52.808632 3289 ts_tablet_manager.cc:1428] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:43:52.808110 3290 consensus_queue.cc:260] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.809777 3290 ts_tablet_manager.cc:1428] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:43:52.899952 3203 raft_consensus.cc:482] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:52.900300 3203 raft_consensus.cc:504] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.901194 3203 leader_election.cc:288] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:52.901876 3124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "534a2e7feb08446e8456dfa658835911" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526" is_pre_election: true
I20260807 19:43:52.901866 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "534a2e7feb08446e8456dfa658835911" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30" is_pre_election: true
I20260807 19:43:52.902146 2972 raft_consensus.cc:2454] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:52.902168 3124 raft_consensus.cc:2454] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 0.
I20260807 19:43:52.902729 2751 leader_election.cc:302] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16, 3c38bb9708a04807881f1ac58062ba30; no voters:
I20260807 19:43:52.903124 3203 raft_consensus.cc:2781] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:52.903223 3203 raft_consensus.cc:482] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:52.903379 3203 raft_consensus.cc:3037] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:52.905866 3203 raft_consensus.cc:504] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.906606 3203 leader_election.cc:288] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Requested vote from peers 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957), 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963)
I20260807 19:43:52.907305 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "534a2e7feb08446e8456dfa658835911" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30"
I20260807 19:43:52.907328 3124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "534a2e7feb08446e8456dfa658835911" candidate_uuid: "0511bae4536d4286bb89b2552e693b16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d83bc9bfc540e399d2e94b5198d526"
I20260807 19:43:52.907634 2972 raft_consensus.cc:3037] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:52.907653 3124 raft_consensus.cc:3037] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:52.909857 3124 raft_consensus.cc:2454] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:52.909991 2972 raft_consensus.cc:2454] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0511bae4536d4286bb89b2552e693b16 in term 1.
I20260807 19:43:52.910439 2751 leader_election.cc:302] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:52.910871 3203 raft_consensus.cc:2781] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:52.911067 3203 raft_consensus.cc:686] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Becoming Leader. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Running, Role: LEADER
I20260807 19:43:52.911466 3203 consensus_queue.cc:237] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:52.914870 2655 catalog_manager.cc:5495] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0511bae4536d4286bb89b2552e693b16 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "0511bae4536d4286bb89b2552e693b16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } health_report { overall_health: UNKNOWN } } }
I20260807 19:43:52.949160 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:52.942229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.0.198.193:38953)
0807 19:43:52.943197 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:52.943202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:52.943243 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:52.943312 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:52.943316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:52.943556 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:52.943684 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:52.944335 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:52.945407 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:52.947087 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:52.947606 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:52.947771 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:52.948296 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:52.948413 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:52.948521 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:52.948707 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 19:43:52.948805 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":394,"threads_started":1}
I20260807 19:43:52.960830 2972 raft_consensus.cc:1261] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:52.960920 3124 raft_consensus.cc:1261] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Refusing update from remote peer 0511bae4536d4286bb89b2552e693b16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:52.961515 3268 consensus_queue.cc:1035] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:52.961946 3203 consensus_queue.cc:1035] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:53.024188 2656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:53.024430 2656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:53.028119 2656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Sending DeleteTablet for 3 replicas of tablet 534a2e7feb08446e8456dfa658835911
I20260807 19:43:53.029212 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 534a2e7feb08446e8456dfa658835911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:53 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60120
I20260807 19:43:53.029359 3095 tablet_service.cc:1514] Processing DeleteTablet for tablet 534a2e7feb08446e8456dfa658835911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:53 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:54674
I20260807 19:43:53.029596 3279 tablet_replica.cc:330] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: stopping tablet replica
I20260807 19:43:53.029676 3278 tablet_replica.cc:330] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: stopping tablet replica
I20260807 19:43:53.029851 3279 raft_consensus.cc:2229] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:53.029897 3278 raft_consensus.cc:2229] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:53.030133 3278 raft_consensus.cc:2258] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:53.030303 3279 raft_consensus.cc:2258] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:53.029340 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 534a2e7feb08446e8456dfa658835911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:53 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45590
I20260807 19:43:53.030710 3277 tablet_replica.cc:330] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: stopping tablet replica
I20260807 19:43:53.030997 3277 raft_consensus.cc:2229] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:53.031251 3277 raft_consensus.cc:2258] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:53.031713 3278 ts_tablet_manager.cc:1905] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:53.031713 3279 ts_tablet_manager.cc:1905] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:53.031582 2656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:53.032418 2656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:53.035046 3277 ts_tablet_manager.cc:1905] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:53.039637 3278 ts_tablet_manager.cc:1918] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:53.039827 3278 log.cc:1196] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/534a2e7feb08446e8456dfa658835911
I20260807 19:43:53.040386 3278 ts_tablet_manager.cc:1939] T 534a2e7feb08446e8456dfa658835911 P 04d83bc9bfc540e399d2e94b5198d526: Deleting consensus metadata
I20260807 19:43:53.041718 2640 catalog_manager.cc:4841] TS 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963): tablet 534a2e7feb08446e8456dfa658835911 (table test-table [id=b50d09f8615b4cfb8970eb740920a6c6]) successfully deleted
I20260807 19:43:53.041950 3277 ts_tablet_manager.cc:1918] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:53.042133 3277 log.cc:1196] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/534a2e7feb08446e8456dfa658835911
I20260807 19:43:53.042305 3279 ts_tablet_manager.cc:1918] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:43:53.042546 3277 ts_tablet_manager.cc:1939] T 534a2e7feb08446e8456dfa658835911 P 3c38bb9708a04807881f1ac58062ba30: Deleting consensus metadata
I20260807 19:43:53.042519 3279 log.cc:1196] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/534a2e7feb08446e8456dfa658835911
I20260807 19:43:53.043012 3279 ts_tablet_manager.cc:1939] T 534a2e7feb08446e8456dfa658835911 P 0511bae4536d4286bb89b2552e693b16: Deleting consensus metadata
I20260807 19:43:53.043982 2641 catalog_manager.cc:4841] TS 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957): tablet 534a2e7feb08446e8456dfa658835911 (table test-table [id=b50d09f8615b4cfb8970eb740920a6c6]) successfully deleted
I20260807 19:43:53.044250 2640 catalog_manager.cc:4841] TS 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953): tablet 534a2e7feb08446e8456dfa658835911 (table test-table [id=b50d09f8615b4cfb8970eb740920a6c6]) successfully deleted
I20260807 19:43:53.048085 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 (DEFAULT_TABLE table=test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:53.048491 3095 tablet_service.cc:1467] Processing CreateTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 (DEFAULT_TABLE table=test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:53.048640 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a1845ed7c94899bc0d5b3dbc32be58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:53.048610 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 (DEFAULT_TABLE table=test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:53.048903 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a1845ed7c94899bc0d5b3dbc32be58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:53.049036 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a1845ed7c94899bc0d5b3dbc32be58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:53.055932 3289 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Bootstrap starting.
I20260807 19:43:53.056013 3290 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Bootstrap starting.
I20260807 19:43:53.056315 3288 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Bootstrap starting.
I20260807 19:43:53.057616 3290 tablet_bootstrap.cc:654] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:53.057833 3289 tablet_bootstrap.cc:654] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:53.058072 3288 tablet_bootstrap.cc:654] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:53.059849 3290 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: No bootstrap required, opened a new log
I20260807 19:43:53.060034 3290 ts_tablet_manager.cc:1397] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:43:53.060892 3290 raft_consensus.cc:348] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.061163 3290 raft_consensus.cc:374] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:53.061249 3290 raft_consensus.cc:729] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Initialized, Role: FOLLOWER
I20260807 19:43:53.061496 3290 consensus_queue.cc:260] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.061759 3288 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: No bootstrap required, opened a new log
I20260807 19:43:53.061930 3288 ts_tablet_manager.cc:1397] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 19:43:53.062048 3290 ts_tablet_manager.cc:1428] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:43:53.063678 3288 raft_consensus.cc:348] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.063944 3288 raft_consensus.cc:374] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:53.064046 3288 raft_consensus.cc:729] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0511bae4536d4286bb89b2552e693b16, State: Initialized, Role: FOLLOWER
I20260807 19:43:53.064499 3288 consensus_queue.cc:260] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.065274 3288 ts_tablet_manager.cc:1428] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 19:43:53.069284 3289 tablet_bootstrap.cc:492] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: No bootstrap required, opened a new log
I20260807 19:43:53.069530 3289 ts_tablet_manager.cc:1397] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.007s
I20260807 19:43:53.070391 3289 raft_consensus.cc:348] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.070680 3289 raft_consensus.cc:374] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:53.070782 3289 raft_consensus.cc:729] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c38bb9708a04807881f1ac58062ba30, State: Initialized, Role: FOLLOWER
I20260807 19:43:53.071058 3289 consensus_queue.cc:260] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.071672 3289 ts_tablet_manager.cc:1428] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:43:53.294572 3294 raft_consensus.cc:482] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:53.294839 3294 raft_consensus.cc:504] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.296667 3294 leader_election.cc:288] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953), 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957)
I20260807 19:43:53.306303 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.297096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.198.195:55051)
0807 19:43:53.298917 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:53.298958 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:53.299232 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:53.299997 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:53.300002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:53.300043 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:53.300548 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.300559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.301869 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.301875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.303755 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.303770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.305286 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.305292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.305514 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.305540 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:43:53.305727 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 19:43:53.305891 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1508,"mutex_wait_us":55,"thread_start_us":1825,"threads_started":1}
I20260807 19:43:53.306354 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.297874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:55051 (local address 127.0.198.194:38957)
0807 19:43:53.299656 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:53.299662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:53.299696 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:43:53.299759 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:53.299763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:53.299818 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:53.299960 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:53.300847 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.301698 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.303970 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.305065 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.305282 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.305359 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:53.305820 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:43:53.306008 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1628,"thread_start_us":1386,"threads_started":1}
I20260807 19:43:53.306350 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.297935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:32967 (local address 127.0.198.193:38953)
0807 19:43:53.298326 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:53.298336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:53.299632 (+ 1296us) server_negotiation.cc:407] Connection header received
0807 19:43:53.299729 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:53.299733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:53.299846 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:53.299987 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:53.300931 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.301842 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.303923 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.304991 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.305191 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.305278 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:53.305990 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 19:43:53.306141 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:43:53.307039 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a1845ed7c94899bc0d5b3dbc32be58" candidate_uuid: "04d83bc9bfc540e399d2e94b5198d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30" is_pre_election: true
I20260807 19:43:53.307276 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.297117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:38953 (local address 127.0.198.195:32967)
0807 19:43:53.299376 (+ 2259us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:53.299395 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:53.299541 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:53.300034 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:53.300039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:53.300067 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:53.300560 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.300570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.302003 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.302007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.303756 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.303769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.305456 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.305463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.305715 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.305765 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:43:53.306632 (+ 867us) client_negotiation.cc:240] Negotiation successful
0807 19:43:53.306777 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2076,"mutex_wait_us":98,"thread_start_us":1223,"threads_started":1}
I20260807 19:43:53.307307 2971 raft_consensus.cc:2454] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04d83bc9bfc540e399d2e94b5198d526 in term 0.
I20260807 19:43:53.308024 3046 leader_election.cc:302] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 3c38bb9708a04807881f1ac58062ba30; no voters:
I20260807 19:43:53.308135 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a1845ed7c94899bc0d5b3dbc32be58" candidate_uuid: "04d83bc9bfc540e399d2e94b5198d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511bae4536d4286bb89b2552e693b16" is_pre_election: true
I20260807 19:43:53.308337 2823 raft_consensus.cc:2454] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04d83bc9bfc540e399d2e94b5198d526 in term 0.
I20260807 19:43:53.308518 3294 raft_consensus.cc:2781] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:53.308648 3294 raft_consensus.cc:482] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:53.308825 3294 raft_consensus.cc:3037] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:53.311280 3294 raft_consensus.cc:504] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.312074 3294 leader_election.cc:288] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [CANDIDATE]: Term 1 election: Requested vote from peers 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953), 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957)
I20260807 19:43:53.312619 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a1845ed7c94899bc0d5b3dbc32be58" candidate_uuid: "04d83bc9bfc540e399d2e94b5198d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0511bae4536d4286bb89b2552e693b16"
I20260807 19:43:53.312758 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a1845ed7c94899bc0d5b3dbc32be58" candidate_uuid: "04d83bc9bfc540e399d2e94b5198d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c38bb9708a04807881f1ac58062ba30"
I20260807 19:43:53.312894 2823 raft_consensus.cc:3037] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:53.313030 2971 raft_consensus.cc:3037] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:53.315186 2823 raft_consensus.cc:2454] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04d83bc9bfc540e399d2e94b5198d526 in term 1.
I20260807 19:43:53.315824 3045 leader_election.cc:302] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d83bc9bfc540e399d2e94b5198d526, 0511bae4536d4286bb89b2552e693b16; no voters:
I20260807 19:43:53.316143 2971 raft_consensus.cc:2454] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04d83bc9bfc540e399d2e94b5198d526 in term 1.
I20260807 19:43:53.316188 3294 raft_consensus.cc:2781] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:53.326508 3294 raft_consensus.cc:686] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 LEADER]: Becoming Leader. State: Replica: 04d83bc9bfc540e399d2e94b5198d526, State: Running, Role: LEADER
I20260807 19:43:53.327085 3294 consensus_queue.cc:237] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } }
I20260807 19:43:53.331159 2656 catalog_manager.cc:5495] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195). New cstate: current_term: 1 leader_uuid: "04d83bc9bfc540e399d2e94b5198d526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04d83bc9bfc540e399d2e94b5198d526" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 32963 } health_report { overall_health: HEALTHY } } }
I20260807 19:43:53.385156 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.377723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.0.198.195:32963)
0807 19:43:53.378500 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:53.378504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:53.378525 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:43:53.378609 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:53.378613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:53.378691 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:53.378847 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:53.379318 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.380313 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.382062 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.382515 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.382618 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.383328 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:53.384354 (+ 1026us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:53.384526 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:53.384621 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:43:53.384704 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":519,"threads_started":1}
I20260807 19:43:53.404700 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.397963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:55723 (local address 127.0.198.194:38957)
0807 19:43:53.398291 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:53.398301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:53.398913 (+ 612us) server_negotiation.cc:407] Connection header received
0807 19:43:53.399224 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:53.399229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:53.399307 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:53.399463 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:53.400353 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.401237 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.402722 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.403175 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.403329 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.403885 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:53.404004 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:53.404125 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:53.404277 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 19:43:53.404368 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260807 19:43:53.404829 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.397950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38957 (local address 127.0.198.195:55723)
0807 19:43:53.398770 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:53.398790 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:53.399046 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:53.399483 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:53.399489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:53.399502 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:53.400036 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.400048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.401395 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.401400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.402549 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.402563 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.403429 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.403436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.403594 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.403666 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:53.404126 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:53.404140 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 19:43:53.404250 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:43:53.404406 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":405,"threads_started":1}
I20260807 19:43:53.415071 2971 raft_consensus.cc:1261] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Refusing update from remote peer 04d83bc9bfc540e399d2e94b5198d526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:53.415472 2823 raft_consensus.cc:1261] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Refusing update from remote peer 04d83bc9bfc540e399d2e94b5198d526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:53.415980 3311 consensus_queue.cc:1035] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c38bb9708a04807881f1ac58062ba30" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:53.416420 3294 consensus_queue.cc:1035] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0511bae4536d4286bb89b2552e693b16" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:53.449958 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.443391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41667 (local address 127.0.198.195:32963)
0807 19:43:53.443582 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:53.443590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:53.443939 (+ 349us) server_negotiation.cc:407] Connection header received
0807 19:43:53.444168 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:53.444173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:53.444285 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:53.444380 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:53.444898 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.445747 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.447776 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.448377 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.448536 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.449180 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:53.449301 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:53.449375 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:53.449685 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 19:43:53.449764 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:43:53.450012 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:53.443089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:32963 (local address 127.0.198.194:41667)
0807 19:43:53.443824 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:53.443840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:53.444007 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:53.444418 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:53.444422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:53.444432 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:43:53.444708 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.444716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.445928 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.445934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.447572 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:53.447609 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:53.448786 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:53.448794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:53.448956 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:53.449000 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:43:53.449411 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:43:53.449423 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:43:53.449522 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:43:53.449659 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":345,"threads_started":1}
W20260807 19:43:53.451488 3258 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:43:55.086848 2655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:55.087105 2655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:55.092036 2655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2b2d7d625234fce803ede7a542f29df: Sending DeleteTablet for 3 replicas of tablet 21a1845ed7c94899bc0d5b3dbc32be58
I20260807 19:43:55.093230 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:55 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45590
I20260807 19:43:55.093307 3095 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:55 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:54674
I20260807 19:43:55.093377 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a1845ed7c94899bc0d5b3dbc32be58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:55 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60120
I20260807 19:43:55.094158 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2733
I20260807 19:43:55.108747 3344 tablet_replica.cc:330] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: stopping tablet replica
I20260807 19:43:55.109084 3344 raft_consensus.cc:2229] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:55.109402 3344 raft_consensus.cc:2258] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:55.110112 3345 tablet_replica.cc:330] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: stopping tablet replica
I20260807 19:43:55.110356 3345 raft_consensus.cc:2229] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:55.110743 3344 ts_tablet_manager.cc:1905] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:55.111048 3345 raft_consensus.cc:2258] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:55.111754 3346 tablet_replica.cc:330] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: stopping tablet replica
I20260807 19:43:55.112071 3346 raft_consensus.cc:2229] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:55.112378 3346 raft_consensus.cc:2258] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:55.112583 3345 ts_tablet_manager.cc:1905] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:55.113428 3346 ts_tablet_manager.cc:1905] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:55.119289 3345 ts_tablet_manager.cc:1918] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:55.119567 3345 log.cc:1196] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/21a1845ed7c94899bc0d5b3dbc32be58
I20260807 19:43:55.120023 3345 ts_tablet_manager.cc:1939] T 21a1845ed7c94899bc0d5b3dbc32be58 P 04d83bc9bfc540e399d2e94b5198d526: Deleting consensus metadata
I20260807 19:43:55.120762 3344 ts_tablet_manager.cc:1918] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:55.120940 3344 log.cc:1196] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/21a1845ed7c94899bc0d5b3dbc32be58
I20260807 19:43:55.121330 2640 catalog_manager.cc:4841] TS 04d83bc9bfc540e399d2e94b5198d526 (127.0.198.195:32963): tablet 21a1845ed7c94899bc0d5b3dbc32be58 (table test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]) successfully deleted
I20260807 19:43:55.121510 3344 ts_tablet_manager.cc:1939] T 21a1845ed7c94899bc0d5b3dbc32be58 P 3c38bb9708a04807881f1ac58062ba30: Deleting consensus metadata
I20260807 19:43:55.122103 3346 ts_tablet_manager.cc:1918] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:43:55.122303 3346 log.cc:1196] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/21a1845ed7c94899bc0d5b3dbc32be58
I20260807 19:43:55.122918 3346 ts_tablet_manager.cc:1939] T 21a1845ed7c94899bc0d5b3dbc32be58 P 0511bae4536d4286bb89b2552e693b16: Deleting consensus metadata
I20260807 19:43:55.123016 2641 catalog_manager.cc:4841] TS 3c38bb9708a04807881f1ac58062ba30 (127.0.198.194:38957): tablet 21a1845ed7c94899bc0d5b3dbc32be58 (table test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]) successfully deleted
I20260807 19:43:55.124279 2640 catalog_manager.cc:4841] TS 0511bae4536d4286bb89b2552e693b16 (127.0.198.193:38953): tablet 21a1845ed7c94899bc0d5b3dbc32be58 (table test-table [id=b6e226e244c14a15bccc2dfa0c3550a1]) successfully deleted
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:43:55.134240 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:55.097998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.0.198.193:38953)
0807 19:43:55.109068 (+ 11070us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:55.109077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:55.109114 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:43:55.109198 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:55.109204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:55.109299 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:55.109450 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:55.112330 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.113760 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.115823 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.116524 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.116679 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:55.127054 (+ 10375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:55.127093 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:55.127116 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:55.127166 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:55.130758 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.131881 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.131891 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.131894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.132079 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.132648 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.132651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.132653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.133051 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:55.133304 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:55.133719 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 19:43:55.133940 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10868,"thread_start_us":11544,"threads_started":1}
W20260807 19:43:55.134982 2869 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55978
W20260807 19:43:55.135686 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2733: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:55.135913 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2733
I20260807 19:43:55.163560 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2878
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:43:55.181623 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:55.164444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.0.198.194:38957)
0807 19:43:55.165032 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:55.165036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:55.165052 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:43:55.165110 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:55.165113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:55.165178 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:55.165280 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:55.166683 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.167655 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.169078 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.169520 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.169644 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:55.174449 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:55.174467 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:55.174485 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:55.174531 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:55.177975 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.179145 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.179158 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.179163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.179438 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.179946 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.179949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.179951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.180348 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:55.180580 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:55.181024 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 19:43:55.181216 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":300,"threads_started":1}
W20260807 19:43:55.182575 3017 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37006
W20260807 19:43:55.183295 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2878: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:55.183537 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2878
I20260807 19:43:55.211102 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3027
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[2611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:43:55.229490 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:55.212089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.0.198.195:32963)
0807 19:43:55.212694 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:55.212699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:55.212718 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:55.212775 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:55.212778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:55.212845 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:55.212941 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:55.214630 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.215781 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.217261 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:55.217744 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:55.217870 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:55.222698 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:55.222718 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:55.222740 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:55.222813 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:55.226137 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.227083 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.227092 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.227094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.227302 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:55.227830 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:55.227833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:55.227835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:55.228222 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:55.228482 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:55.228867 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:43:55.229074 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":327,"threads_started":1}
W20260807 19:43:55.230415 3166 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57490
W20260807 19:43:55.231256 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3027: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:55.231642 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3027
I20260807 19:43:55.260191 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 2624
I20260807 19:43:55.261597 2701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:55.387290 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 2624
2026-08-07T19:43:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9282 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[3354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[3354](info): set up 2 sockets
Aug 07 19:43:55 dist-test-slave-btw8 krb5kdc[3354](info): commencing operation
krb5kdc: starting...
W20260807 19:43:56.601341 795 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.005s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:43:56 dist-test-slave-btw8 krb5kdc[3354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:43:56.879842 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34591
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:57.248361 3367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:57.248832 3367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:57.249023 3367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:57.257756 3367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:57.257947 3367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:57.258066 3367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:57.258168 3367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:57.258261 3367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:57.269330 3367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:57.270925 3367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:57.272436 3367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:57.286185 3372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:57.286158 3375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:57.286134 3373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:57.286715 3367 server_base.cc:1034] running on GCE node
I20260807 19:43:57.287817 3367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:57.289529 3367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:57.290815 3367 hybrid_clock.cc:648] HybridClock initialized: now 1786131837290766 us; error 43 us; skew 500 ppm
I20260807 19:43:57.291219 3367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:57 dist-test-slave-btw8 krb5kdc[3354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:57.294792 3367 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:43:57.296571 3367 webserver.cc:460] Webserver started at http://127.0.198.254:35649/ using document root <none> and password file <none>
I20260807 19:43:57.297340 3367 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:57.297504 3367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:57.297852 3367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:57.301178 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "3c77cc69bd0b40e984de9bdd29419122"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "389e6fc006b7e52dcc3d2ce9d5a89a19"
server_key_iv: "1bc3a8ad12993bb6769d8888a4d27259"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.302055 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "3c77cc69bd0b40e984de9bdd29419122"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "389e6fc006b7e52dcc3d2ce9d5a89a19"
server_key_iv: "1bc3a8ad12993bb6769d8888a4d27259"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.308220 3367 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260807 19:43:57.312264 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:57.313941 3367 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:43:57.314162 3367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "3c77cc69bd0b40e984de9bdd29419122"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "389e6fc006b7e52dcc3d2ce9d5a89a19"
server_key_iv: "1bc3a8ad12993bb6769d8888a4d27259"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.314409 3367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:57.329730 3367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:57.335147 3367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:57.335572 3367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:57.356120 3367 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:34591
I20260807 19:43:57.356153 3436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:34591 every 8 connection(s)
I20260807 19:43:57.357692 3367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:57.361666 3437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:57.368271 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3367
I20260807 19:43:57.368578 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:43:57.369390 795 external_mini_cluster.cc:1428] Setting key 12b445ea2c9dcf07e61706c3ff82b033
I20260807 19:43:57.373749 3437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122: Bootstrap starting.
I20260807 19:43:57.377854 3437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:57.379096 3437 log.cc:824] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:57.382709 3437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122: No bootstrap required, opened a new log
Aug 07 19:43:57 dist-test-slave-btw8 krb5kdc[3354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:57.393184 3437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } }
I20260807 19:43:57.393677 3437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:57.393801 3437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c77cc69bd0b40e984de9bdd29419122, State: Initialized, Role: FOLLOWER
I20260807 19:43:57.394692 3437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } }
I20260807 19:43:57.395063 3437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:57.395416 3437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:57.395689 3437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:57.399695 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:57.371947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.0.198.254:34591)
0807 19:43:57.372887 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:57.372901 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:57.372946 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:43:57.375029 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:57.375057 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:57.375631 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:57.376019 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:57.378153 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:57.379853 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:57.381681 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:57.382431 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:57.382616 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:57.388379 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:57.388390 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:57.388414 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:57.388470 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:57.392782 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:57.393946 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:57.393955 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:57.393964 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:57.394297 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:57.394905 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:57.394911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:57.394916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:57.396163 (+ 1247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:57.396517 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:57.397050 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 19:43:57.397623 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":467,"threads_started":1}
I20260807 19:43:57.399719 3437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } }
I20260807 19:43:57.400678 3437 leader_election.cc:302] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c77cc69bd0b40e984de9bdd29419122; no voters:
I20260807 19:43:57.401412 3437 leader_election.cc:288] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:57.401571 3443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:57.402918 3443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [term 1 LEADER]: Becoming Leader. State: Replica: 3c77cc69bd0b40e984de9bdd29419122, State: Running, Role: LEADER
I20260807 19:43:57.403607 3443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } }
I20260807 19:43:57.404075 3437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:57.406566 3445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c77cc69bd0b40e984de9bdd29419122. Latest consensus state: current_term: 1 leader_uuid: "3c77cc69bd0b40e984de9bdd29419122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } } }
I20260807 19:43:57.407023 3445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:57.407552 3444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c77cc69bd0b40e984de9bdd29419122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77cc69bd0b40e984de9bdd29419122" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34591 } } }
I20260807 19:43:57.407889 3444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:57.413239 3450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:57.420702 3450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:57.432713 3450 catalog_manager.cc:1347] Generated new cluster ID: 7430c876984e442c99f486a02b6c4d07
I20260807 19:43:57.432873 3450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:57.441336 3450 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:57.443090 3450 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:57.451977 3450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c77cc69bd0b40e984de9bdd29419122: Generated new TSK 0
I20260807 19:43:57.452979 3450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:43:57.525049 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:57.927894 3465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:57.928251 3465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:57.928490 3465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:57.937522 3465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:57.937681 3465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:57.937907 3465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:43:57.949937 3465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:57.951771 3465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:57.953351 3465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:57.970368 3470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:57.970281 3471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:57.970300 3473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:57.971298 3465 server_base.cc:1034] running on GCE node
I20260807 19:43:57.972079 3465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:57.973147 3465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:57.974407 3465 hybrid_clock.cc:648] HybridClock initialized: now 1786131837974361 us; error 76 us; skew 500 ppm
I20260807 19:43:57.974762 3465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:57 dist-test-slave-btw8 krb5kdc[3354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:57.978317 3465 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:43:57.980019 3465 webserver.cc:460] Webserver started at http://127.0.198.193:36965/ using document root <none> and password file <none>
I20260807 19:43:57.980786 3465 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:57.980947 3465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:57.981297 3465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:57.984464 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "6d04871b4ef645f8810e0c4f02f324da"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "b5e1a80dcf87d8347ddff49c843ddb41"
server_key_iv: "fadc8c10f7ff1fd2b236f26d9046588c"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.985353 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "6d04871b4ef645f8810e0c4f02f324da"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "b5e1a80dcf87d8347ddff49c843ddb41"
server_key_iv: "fadc8c10f7ff1fd2b236f26d9046588c"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.991436 3465 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:43:57.995436 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:57.996975 3465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:43:57.997247 3465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "6d04871b4ef645f8810e0c4f02f324da"
format_stamp: "Formatted at 2026-08-07 19:43:57 on dist-test-slave-btw8"
server_key: "b5e1a80dcf87d8347ddff49c843ddb41"
server_key_iv: "fadc8c10f7ff1fd2b236f26d9046588c"
server_key_version: "encryptionkey@0"
I20260807 19:43:57.997496 3465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:58.028589 3465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:58.033694 3465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:58.034080 3465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:58.035283 3465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:58.036870 3465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:58.037014 3465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:58.037176 3465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:58.037267 3465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:58.073984 3465 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:34035
I20260807 19:43:58.074056 3604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:34035 every 8 connection(s)
I20260807 19:43:58.075580 3465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:58.084438 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3465
I20260807 19:43:58.084719 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:58.085541 795 external_mini_cluster.cc:1428] Setting key 9fcb8227e5adf21e57f5deb6ae17f16b
Aug 07 19:43:58 dist-test-slave-btw8 krb5kdc[3354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:58.100773 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:58.078655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:36615 (local address 127.0.198.254:34591)
0807 19:43:58.079245 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:58.079253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:58.080391 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 19:43:58.082414 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:58.082420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:58.082526 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:58.082664 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:58.084516 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.085376 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.087327 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.087888 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.088043 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:58.092452 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:58.092461 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:58.092469 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:58.092522 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:58.096384 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:58.097834 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:58.097840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:58.097842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:58.098009 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:58.098609 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:58.098613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:58.098615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:58.098931 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:58.099061 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:58.100179 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0807 19:43:58.100397 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":285,"threads_started":1}
I20260807 19:43:58.102344 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:58.079239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34591 (local address 127.0.198.193:36615)
0807 19:43:58.080174 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:58.080234 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:58.082066 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:58.082946 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:58.082955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:58.083658 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:58.084301 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:58.084320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.085553 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.085564 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:58.087114 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:58.087129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.088150 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.088156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:58.088404 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:58.089295 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:58.089322 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:58.092130 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:58.096687 (+ 4557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:58.096696 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:58.096721 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:58.097587 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:58.098158 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:58.098164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:58.098169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:58.098451 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:58.099083 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:58.099096 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:58.099680 (+ 584us) client_negotiation.cc:769] Sending connection context
0807 19:43:58.100020 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 19:43:58.100685 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":340,"threads_started":1}
I20260807 19:43:58.104709 3605 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34591
I20260807 19:43:58.105073 3605 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:58.106163 3605 heartbeater.cc:511] Master 127.0.198.254:34591 requested a full tablet report, sending...
I20260807 19:43:58.109148 3400 ts_manager.cc:194] Registered new tserver with Master: 6d04871b4ef645f8810e0c4f02f324da (127.0.198.193:34035)
I20260807 19:43:58.111738 3400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:36615
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:43:58.151034 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:58.519881 3611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:58.520252 3611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:58.520434 3611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:58.529029 3611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:58.529183 3611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:58.529393 3611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:43:58.540259 3611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:58.541725 3611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:58.543191 3611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:58.557933 3619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:58.557988 3618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:58.557911 3621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:58.559134 3611 server_base.cc:1034] running on GCE node
I20260807 19:43:58.559890 3611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:58.561026 3611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:58.562271 3611 hybrid_clock.cc:648] HybridClock initialized: now 1786131838562219 us; error 38 us; skew 500 ppm
I20260807 19:43:58.562692 3611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:58 dist-test-slave-btw8 krb5kdc[3354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:58.566606 3611 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:43:58.568584 3611 webserver.cc:460] Webserver started at http://127.0.198.194:42011/ using document root <none> and password file <none>
I20260807 19:43:58.569443 3611 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:58.569567 3611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:58.569944 3611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:58.573596 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "598f2db443704fca8dcd1d1218d2b43d"
format_stamp: "Formatted at 2026-08-07 19:43:58 on dist-test-slave-btw8"
server_key: "29238165da7ab958b89d5755d25d67bc"
server_key_iv: "547ba3891f1423d4c3197b039495ac84"
server_key_version: "encryptionkey@0"
I20260807 19:43:58.574554 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "598f2db443704fca8dcd1d1218d2b43d"
format_stamp: "Formatted at 2026-08-07 19:43:58 on dist-test-slave-btw8"
server_key: "29238165da7ab958b89d5755d25d67bc"
server_key_iv: "547ba3891f1423d4c3197b039495ac84"
server_key_version: "encryptionkey@0"
I20260807 19:43:58.581060 3611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:43:58.585417 3629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:58.587179 3611 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 19:43:58.587466 3611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "598f2db443704fca8dcd1d1218d2b43d"
format_stamp: "Formatted at 2026-08-07 19:43:58 on dist-test-slave-btw8"
server_key: "29238165da7ab958b89d5755d25d67bc"
server_key_iv: "547ba3891f1423d4c3197b039495ac84"
server_key_version: "encryptionkey@0"
I20260807 19:43:58.587702 3611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:58.604795 3611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:58.609720 3611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:58.610056 3611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:58.611152 3611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:58.612782 3611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:58.612859 3611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:58.612984 3611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:58.613070 3611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:58.651902 3611 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:38965
I20260807 19:43:58.651945 3748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:38965 every 8 connection(s)
I20260807 19:43:58.653419 3611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:58.658921 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3611
I20260807 19:43:58.659166 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:58.659760 795 external_mini_cluster.cc:1428] Setting key 0309ab4ff050937292b77d7ff8774d96
Aug 07 19:43:58 dist-test-slave-btw8 krb5kdc[3354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:58.679467 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:58.656587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:33563 (local address 127.0.198.254:34591)
0807 19:43:58.657136 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:58.657141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:58.658086 (+ 945us) server_negotiation.cc:407] Connection header received
0807 19:43:58.660207 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:58.660221 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:58.660325 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:58.660448 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:58.662539 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.663300 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.665994 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.666480 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.666590 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:58.671698 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:58.671707 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:58.671713 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:58.671767 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:58.675669 (+ 3902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:58.676607 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:58.676615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:58.676618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:58.676790 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:58.677268 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:58.677276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:58.677282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:58.677718 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:58.677882 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:58.678638 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 19:43:58.678928 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":324,"threads_started":1}
I20260807 19:43:58.680305 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:58.657067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34591 (local address 127.0.198.194:33563)
0807 19:43:58.657905 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:58.657950 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:58.659906 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:58.660843 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:58.660857 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:58.661501 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:58.662344 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:58.662371 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.663499 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.663508 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:58.665805 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:58.665827 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:58.666771 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:58.666781 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:58.667189 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:58.668117 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:58.668155 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:58.671249 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:58.675895 (+ 4646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:58.675904 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:58.675922 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:58.676443 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:58.676919 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:58.676922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:58.676925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:58.677143 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:58.677847 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:58.677854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:58.678208 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:43:58.678497 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 19:43:58.678965 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":314,"threads_started":1}
I20260807 19:43:58.682075 3749 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34591
I20260807 19:43:58.682440 3749 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:58.683208 3749 heartbeater.cc:511] Master 127.0.198.254:34591 requested a full tablet report, sending...
I20260807 19:43:58.684914 3400 ts_manager.cc:194] Registered new tserver with Master: 598f2db443704fca8dcd1d1218d2b43d (127.0.198.194:38965)
I20260807 19:43:58.685781 3400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:33563
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:43:58.719933 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:59.087241 3755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:59.087670 3755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:59.087841 3755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:59.095847 3755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:59.095970 3755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:59.096160 3755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:43:59.107259 3755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:59.108776 3755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:59.110246 3755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:43:59.115913 3605 heartbeater.cc:503] Master 127.0.198.254:34591 was elected leader, sending a full tablet report...
W20260807 19:43:59.125821 3761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:59.125923 3760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:59.125898 3763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:59.126948 3755 server_base.cc:1034] running on GCE node
I20260807 19:43:59.127914 3755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:59.129231 3755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:59.130510 3755 hybrid_clock.cc:648] HybridClock initialized: now 1786131839130423 us; error 71 us; skew 500 ppm
I20260807 19:43:59.131022 3755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:59 dist-test-slave-btw8 krb5kdc[3354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:59.134984 3755 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:43:59.137009 3755 webserver.cc:460] Webserver started at http://127.0.198.195:44711/ using document root <none> and password file <none>
I20260807 19:43:59.137869 3755 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:59.137988 3755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:59.138376 3755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:59.141677 3755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "96fcef30e95c468381cf4447e50503f9"
format_stamp: "Formatted at 2026-08-07 19:43:59 on dist-test-slave-btw8"
server_key: "8f6ba236698980c1dc1c935756e652a6"
server_key_iv: "4cd944b52b818fe0e53eff0aa4756c39"
server_key_version: "encryptionkey@0"
I20260807 19:43:59.142972 3755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "96fcef30e95c468381cf4447e50503f9"
format_stamp: "Formatted at 2026-08-07 19:43:59 on dist-test-slave-btw8"
server_key: "8f6ba236698980c1dc1c935756e652a6"
server_key_iv: "4cd944b52b818fe0e53eff0aa4756c39"
server_key_version: "encryptionkey@0"
I20260807 19:43:59.149870 3755 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:43:59.154373 3770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:59.156266 3755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:43:59.156515 3755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "96fcef30e95c468381cf4447e50503f9"
format_stamp: "Formatted at 2026-08-07 19:43:59 on dist-test-slave-btw8"
server_key: "8f6ba236698980c1dc1c935756e652a6"
server_key_iv: "4cd944b52b818fe0e53eff0aa4756c39"
server_key_version: "encryptionkey@0"
I20260807 19:43:59.156787 3755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:59.171140 3755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:59.176467 3755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:59.176833 3755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:59.177938 3755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:59.179541 3755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:59.179649 3755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:59.180109 3755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:59.180200 3755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:59.215998 3755 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36337
I20260807 19:43:59.216035 3889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36337 every 8 connection(s)
I20260807 19:43:59.217434 3755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:59.218518 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3755
I20260807 19:43:59.218731 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:59.219504 795 external_mini_cluster.cc:1428] Setting key a541881c43a3aaebf636b97d7ccc788c
Aug 07 19:43:59 dist-test-slave-btw8 krb5kdc[3354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:43:59.242277 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:59.221024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:47445 (local address 127.0.198.254:34591)
0807 19:43:59.221628 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:59.221634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:59.223222 (+ 1588us) server_negotiation.cc:407] Connection header received
0807 19:43:59.225085 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:59.225091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:59.225216 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:59.225371 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:59.227212 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:59.228151 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:59.229973 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:59.230430 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:59.230598 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:59.235124 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:59.235129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:59.235133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:59.235174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:59.238627 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:59.239620 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:59.239626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:59.239629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:59.239774 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:59.240278 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:59.240283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:59.240287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:59.240592 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:59.240784 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:59.241575 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 19:43:59.241863 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":289,"threads_started":1}
I20260807 19:43:59.243464 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:59.221755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34591 (local address 127.0.198.195:47445)
0807 19:43:59.222714 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:59.222789 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:59.224727 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:59.225673 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:59.225682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:59.226315 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:59.226976 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:59.226996 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:59.228377 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:59.228381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:59.229771 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:59.229787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:59.230720 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:59.230729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:59.231011 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:59.231882 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:59.231909 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:59.234829 (+ 2920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:59.238844 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:59.238856 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:59.238881 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:59.239477 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:59.239909 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:59.239914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:59.239918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:59.240141 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:59.240748 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:59.240757 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:59.241166 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 19:43:59.241471 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 19:43:59.242071 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":389,"threads_started":1}
I20260807 19:43:59.245298 3890 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34591
I20260807 19:43:59.245646 3890 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:59.246471 3890 heartbeater.cc:511] Master 127.0.198.254:34591 requested a full tablet report, sending...
I20260807 19:43:59.248286 3400 ts_manager.cc:194] Registered new tserver with Master: 96fcef30e95c468381cf4447e50503f9 (127.0.198.195:36337)
I20260807 19:43:59.249158 3400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:47445
I20260807 19:43:59.249866 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:43:59.295934 3892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:59.292976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.0.198.254:34591)
0807 19:43:59.293250 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:59.293260 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:59.293759 (+ 499us) server_negotiation.cc:407] Connection header received
0807 19:43:59.293901 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:59.293906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:59.293979 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:59.294070 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:59.295576 (+ 1506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41660: BlockingRecv error: recv got EOF from 127.0.0.1:41660 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:43:59.299656 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3465
W20260807 19:43:59.303589 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3465: Not authorized: Client connection negotiation failed: client connection to 127.0.198.193:34035: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:43:59.303885 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3465
I20260807 19:43:59.324592 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3611
W20260807 19:43:59.328894 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3611: Not authorized: Client connection negotiation failed: client connection to 127.0.198.194:38965: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:43:59.329108 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3611
W20260807 19:43:59.329188 3904 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:59.326059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.0.198.194:38965)
0807 19:43:59.327068 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:59.327080 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:59.327121 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:59.327217 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:59.327228 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:59.327634 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:59.327870 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:59.328773 (+ 903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34700: BlockingRecv error: recv got EOF from 127.0.0.1:34700 (error 108)
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":579,"threads_started":1}
I20260807 19:43:59.348347 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3755
W20260807 19:43:59.352232 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3755: Not authorized: Client connection negotiation failed: client connection to 127.0.198.195:36337: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:43:59.352392 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3755
I20260807 19:43:59.370826 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3367
I20260807 19:43:59.372318 3432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:59.480142 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3367
2026-08-07T19:43:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4089 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:59 dist-test-slave-btw8 krb5kdc[3909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:59 dist-test-slave-btw8 krb5kdc[3909](info): set up 2 sockets
Aug 07 19:43:59 dist-test-slave-btw8 krb5kdc[3909](info): commencing operation
krb5kdc: starting...
W20260807 19:44:00.657810 795 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:00 dist-test-slave-btw8 krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:00.932374 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33549
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:01.295011 3923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:01.295462 3923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:01.295674 3923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:01.303589 3923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:01.303750 3923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:01.303833 3923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:01.303932 3923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:01.304004 3923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:01.314610 3923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:01.316166 3923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:01.317691 3923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:01.334507 3931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:01.334475 3929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:01.336421 3928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:01.338394 3923 server_base.cc:1034] running on GCE node
I20260807 19:44:01.339680 3923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:01.341838 3923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:01.343578 3923 hybrid_clock.cc:648] HybridClock initialized: now 1786131841343503 us; error 95 us; skew 500 ppm
I20260807 19:44:01.344146 3923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:01 dist-test-slave-btw8 krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:01.355477 3923 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:01.357631 3923 webserver.cc:460] Webserver started at http://127.0.198.254:38169/ using document root <none> and password file <none>
I20260807 19:44:01.358511 3923 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:01.358659 3923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:01.359046 3923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:01.362872 3923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "c4e615551ea74b738ba862ed1521e277"
format_stamp: "Formatted at 2026-08-07 19:44:01 on dist-test-slave-btw8"
server_key: "8b92c0de05f18b3e83efd0bd49a6ff3d"
server_key_iv: "56006cc2ce43d32bbce0234857757769"
server_key_version: "encryptionkey@0"
I20260807 19:44:01.364002 3923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "c4e615551ea74b738ba862ed1521e277"
format_stamp: "Formatted at 2026-08-07 19:44:01 on dist-test-slave-btw8"
server_key: "8b92c0de05f18b3e83efd0bd49a6ff3d"
server_key_iv: "56006cc2ce43d32bbce0234857757769"
server_key_version: "encryptionkey@0"
I20260807 19:44:01.370677 3923 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:44:01.375779 3939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:01.377948 3923 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:44:01.378144 3923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "c4e615551ea74b738ba862ed1521e277"
format_stamp: "Formatted at 2026-08-07 19:44:01 on dist-test-slave-btw8"
server_key: "8b92c0de05f18b3e83efd0bd49a6ff3d"
server_key_iv: "56006cc2ce43d32bbce0234857757769"
server_key_version: "encryptionkey@0"
I20260807 19:44:01.378365 3923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:01.409793 3923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:01.415109 3923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:01.415453 3923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:01.439196 3923 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33549
I20260807 19:44:01.439237 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33549 every 8 connection(s)
I20260807 19:44:01.440968 3923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:01.442340 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 3923
I20260807 19:44:01.442586 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:01.443554 795 external_mini_cluster.cc:1428] Setting key a1b8eaf42fdba114a9c5fa97638cd517
I20260807 19:44:01.446691 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:44:01 dist-test-slave-btw8 krb5kdc[3909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:01.462603 3996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277: Bootstrap starting.
I20260807 19:44:01.466605 3996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:01.468369 3996 log.cc:824] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:01.471817 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:01.445980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.0.198.254:33549)
0807 19:44:01.447339 (+ 1359us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:01.447416 (+ 77us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:01.447477 (+ 61us) server_negotiation.cc:407] Connection header received
0807 19:44:01.449395 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:01.449422 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:01.449942 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:01.450378 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:01.452855 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:01.454646 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:01.456776 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:01.457582 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:01.457769 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:01.462907 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:01.462918 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:01.462940 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:01.462991 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:01.466398 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:01.467697 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:01.467707 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:01.467718 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:01.467954 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:01.468587 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:01.468597 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:01.468600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:01.469262 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:01.469646 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:01.470202 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 19:44:01.470674 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":776,"threads_started":1}
I20260807 19:44:01.472219 3996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277: No bootstrap required, opened a new log
I20260807 19:44:01.476529 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:44:01.481129 3996 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } }
I20260807 19:44:01.481622 3996 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:01.481750 3996 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e615551ea74b738ba862ed1521e277, State: Initialized, Role: FOLLOWER
I20260807 19:44:01.482386 3996 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } }
I20260807 19:44:01.482676 3996 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:01.482823 3996 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:01.483040 3996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:01.485853 3996 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } }
I20260807 19:44:01.486503 3996 leader_election.cc:302] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e615551ea74b738ba862ed1521e277; no voters:
I20260807 19:44:01.487430 3996 leader_election.cc:288] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:01.487557 4002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:01.490231 4002 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [term 1 LEADER]: Becoming Leader. State: Replica: c4e615551ea74b738ba862ed1521e277, State: Running, Role: LEADER
I20260807 19:44:01.490866 4002 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } }
I20260807 19:44:01.491952 3996 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:01.494481 4005 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e615551ea74b738ba862ed1521e277. Latest consensus state: current_term: 1 leader_uuid: "c4e615551ea74b738ba862ed1521e277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } } }
I20260807 19:44:01.494395 4003 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e615551ea74b738ba862ed1521e277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e615551ea74b738ba862ed1521e277" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33549 } } }
I20260807 19:44:01.494837 4003 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:01.494837 4005 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:01.509507 4008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:01.516213 4008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:01.529491 4008 catalog_manager.cc:1347] Generated new cluster ID: de379c3145c7433e8fdb1448194a2bfa
I20260807 19:44:01.529824 4008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:01.541545 4008 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:01.543651 4008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:01.549248 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:01.540108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.0.198.254:33549)
0807 19:44:01.540311 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:01.540318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:01.540929 (+ 611us) server_negotiation.cc:407] Connection header received
0807 19:44:01.541251 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:01.541258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:01.541348 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:01.541459 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:01.543012 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:01.543970 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:01.545348 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:01.545817 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:01.546156 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:01.547488 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:01.547495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:01.547500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:01.548190 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:01.548241 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:01.548274 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:01.548412 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:01.548573 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 19:44:01.548735 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:44:01.555402 4008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e615551ea74b738ba862ed1521e277: Generated new TSK 0
I20260807 19:44:01.556625 4008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:44:01.573057 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 3923
I20260807 19:44:01.574726 3991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:01.678021 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 3923
2026-08-07T19:44:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2191 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:01 dist-test-slave-btw8 krb5kdc[4033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:01 dist-test-slave-btw8 krb5kdc[4033](info): set up 2 sockets
Aug 07 19:44:01 dist-test-slave-btw8 krb5kdc[4033](info): commencing operation
krb5kdc: starting...
W20260807 19:44:02.893005 795 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:03 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:03.176780 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39279
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:03.539312 4046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:03.539774 4046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:03.539938 4046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:03.547683 4046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:03.547806 4046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:03.547892 4046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:03.548005 4046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:03.548071 4046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:03.559026 4046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:03.560619 4046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:03.562031 4046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:03.575428 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:03.576009 4051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:03.576045 4054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:03.576653 4046 server_base.cc:1034] running on GCE node
I20260807 19:44:03.577634 4046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:03.579277 4046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:03.580567 4046 hybrid_clock.cc:648] HybridClock initialized: now 1786131843580526 us; error 35 us; skew 500 ppm
I20260807 19:44:03.580974 4046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:03 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:03.585237 4046 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:03.587690 4046 webserver.cc:460] Webserver started at http://127.0.198.254:40711/ using document root <none> and password file <none>
I20260807 19:44:03.588521 4046 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:03.588718 4046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:03.589078 4046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:03.592593 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "c8d17a70207b40fdb0d0576d2eb75c65"
format_stamp: "Formatted at 2026-08-07 19:44:03 on dist-test-slave-btw8"
server_key: "313819eb5a44967d7ecb8aea1ce63853"
server_key_iv: "ca2207d7b1ed13b6a52143ed766f8b86"
server_key_version: "encryptionkey@0"
I20260807 19:44:03.593489 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "c8d17a70207b40fdb0d0576d2eb75c65"
format_stamp: "Formatted at 2026-08-07 19:44:03 on dist-test-slave-btw8"
server_key: "313819eb5a44967d7ecb8aea1ce63853"
server_key_iv: "ca2207d7b1ed13b6a52143ed766f8b86"
server_key_version: "encryptionkey@0"
I20260807 19:44:03.599607 4046 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:44:03.604017 4063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:03.605715 4046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:44:03.605981 4046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "c8d17a70207b40fdb0d0576d2eb75c65"
format_stamp: "Formatted at 2026-08-07 19:44:03 on dist-test-slave-btw8"
server_key: "313819eb5a44967d7ecb8aea1ce63853"
server_key_iv: "ca2207d7b1ed13b6a52143ed766f8b86"
server_key_version: "encryptionkey@0"
I20260807 19:44:03.606244 4046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:03.636811 4046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:03.641978 4046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:03.642326 4046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:03.663029 4046 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:39279
I20260807 19:44:03.663065 4120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:39279 every 8 connection(s)
I20260807 19:44:03.664690 4046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:03.666749 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4046
I20260807 19:44:03.667003 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:03.667956 795 external_mini_cluster.cc:1428] Setting key 1b1233c1706ebc5754e1a0c036cc1279
I20260807 19:44:03.670379 4121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:44:03 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:03.685787 4121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65: Bootstrap starting.
I20260807 19:44:03.689502 4121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:03.690737 4121 log.cc:824] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:03.694705 4121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65: No bootstrap required, opened a new log
I20260807 19:44:03.695971 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:03.670344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.0.198.254:39279)
0807 19:44:03.671412 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:03.671429 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:03.671484 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:44:03.673191 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:03.673214 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:03.673629 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:03.673993 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:03.676065 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:03.677694 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:03.679646 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:03.680446 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:03.680634 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:03.685839 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:03.685850 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:03.685868 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:03.685915 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:03.689502 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:03.690456 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:03.690464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:03.690472 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:03.690681 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:03.691196 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:03.691207 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:03.691209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:03.692240 (+ 1031us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:03.692542 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:03.693889 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0807 19:44:03.694393 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":485,"threads_started":1}
I20260807 19:44:03.703020 4121 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } }
I20260807 19:44:03.703506 4121 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:03.703591 4121 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d17a70207b40fdb0d0576d2eb75c65, State: Initialized, Role: FOLLOWER
I20260807 19:44:03.704183 4121 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } }
I20260807 19:44:03.704429 4121 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:03.704535 4121 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:03.704663 4121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:03.706875 4121 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } }
I20260807 19:44:03.707531 4121 leader_election.cc:302] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8d17a70207b40fdb0d0576d2eb75c65; no voters:
I20260807 19:44:03.708305 4121 leader_election.cc:288] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:03.708573 4126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:03.709357 4126 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [term 1 LEADER]: Becoming Leader. State: Replica: c8d17a70207b40fdb0d0576d2eb75c65, State: Running, Role: LEADER
I20260807 19:44:03.709964 4126 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } }
I20260807 19:44:03.710623 4121 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:03.713765 4128 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8d17a70207b40fdb0d0576d2eb75c65. Latest consensus state: current_term: 1 leader_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } } }
I20260807 19:44:03.713749 4127 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d17a70207b40fdb0d0576d2eb75c65" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39279 } } }
I20260807 19:44:03.714308 4128 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:03.714314 4127 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:03.716591 4132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:03.722887 4132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:03.733265 4132 catalog_manager.cc:1347] Generated new cluster ID: de86ad87a37f4fd6bf63993735fbbc7e
I20260807 19:44:03.733423 4132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:03.752770 4132 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:03.754854 4132 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:03.764724 4132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8d17a70207b40fdb0d0576d2eb75c65: Generated new TSK 0
I20260807 19:44:03.765992 4132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:44:03.831944 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:04.209100 4147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:04.209419 4147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:04.209524 4147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:04.218302 4147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:04.218442 4147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:04.218568 4147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:04.229558 4147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:04.230960 4147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:04.232369 4147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:04.247761 4153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:04.247661 4152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:04.247779 4155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:04.248941 4147 server_base.cc:1034] running on GCE node
I20260807 19:44:04.249676 4147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:04.250984 4147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:04.252317 4147 hybrid_clock.cc:648] HybridClock initialized: now 1786131844252265 us; error 45 us; skew 500 ppm
I20260807 19:44:04.252763 4147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:04 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:04.256934 4147 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:44:04.258953 4147 webserver.cc:460] Webserver started at http://127.0.198.193:44005/ using document root <none> and password file <none>
I20260807 19:44:04.259864 4147 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:04.260057 4147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:04.260421 4147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:04.264034 4147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "8ec456f4e4a8488a847c8453d75b54ae"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "d4f552b3bef1b46c7e118d1c6cc6d6bd"
server_key_iv: "3bec0b7a5a1c690f89e80032080bb370"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.265049 4147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "8ec456f4e4a8488a847c8453d75b54ae"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "d4f552b3bef1b46c7e118d1c6cc6d6bd"
server_key_iv: "3bec0b7a5a1c690f89e80032080bb370"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.271862 4147 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:44:04.276289 4163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.277927 4147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:44:04.278169 4147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "8ec456f4e4a8488a847c8453d75b54ae"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "d4f552b3bef1b46c7e118d1c6cc6d6bd"
server_key_iv: "3bec0b7a5a1c690f89e80032080bb370"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.278434 4147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:04.299965 4147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:04.304461 4147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:04.304786 4147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:04.306015 4147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:04.307583 4147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:04.307690 4147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.307847 4147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:04.307920 4147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.349148 4147 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:41551
I20260807 19:44:04.349191 4284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:41551 every 8 connection(s)
I20260807 19:44:04.350644 4147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:04.361083 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4147
I20260807 19:44:04.361318 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:04.362247 795 external_mini_cluster.cc:1428] Setting key fedf789994db9e46543ba73646ecfc97
Aug 07 19:44:04 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:04.375176 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:04.353555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:60131 (local address 127.0.198.254:39279)
0807 19:44:04.354140 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:04.354148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:04.355023 (+ 875us) server_negotiation.cc:407] Connection header received
0807 19:44:04.356950 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:04.356955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:04.357069 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:04.357191 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:04.358851 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.359722 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.361553 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.362253 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.362422 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:04.366875 (+ 4453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:04.366886 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:04.366894 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:04.366952 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:04.370992 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:04.372209 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:04.372217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:04.372223 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:04.372419 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:04.373062 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:04.373070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:04.373075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:04.373507 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:04.373689 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:04.374669 (+ 980us) server_negotiation.cc:299] Negotiation successful
0807 19:44:04.374899 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":284,"threads_started":1}
I20260807 19:44:04.376986 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:04.354036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39279 (local address 127.0.198.193:60131)
0807 19:44:04.354844 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:04.354884 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:04.356624 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:04.357412 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:04.357421 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:04.358037 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:04.358633 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:04.358652 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.359900 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.359904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:04.361350 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:04.361389 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.362553 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.362560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:04.362797 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:04.363813 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:04.363845 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:04.366524 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:04.371255 (+ 4731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:04.371265 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:04.371301 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:04.372042 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:04.372660 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:04.372665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:04.372668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:04.372912 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:04.373677 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:04.373685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:04.374126 (+ 441us) client_negotiation.cc:769] Sending connection context
0807 19:44:04.374538 (+ 412us) client_negotiation.cc:240] Negotiation successful
0807 19:44:04.375250 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":293,"threads_started":1}
I20260807 19:44:04.379088 4286 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39279
I20260807 19:44:04.379525 4286 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:04.380411 4286 heartbeater.cc:511] Master 127.0.198.254:39279 requested a full tablet report, sending...
I20260807 19:44:04.383174 4082 ts_manager.cc:194] Registered new tserver with Master: 8ec456f4e4a8488a847c8453d75b54ae (127.0.198.193:41551)
I20260807 19:44:04.385066 4082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:60131
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:44:04.428122 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:04.794363 4292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:04.794668 4292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:04.794791 4292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:04.802385 4292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:04.802524 4292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:04.802644 4292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:04.813442 4292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:04.814987 4292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:04.816442 4292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:04.832003 4302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:04.832249 4298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:04.832017 4300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:04.833012 4292 server_base.cc:1034] running on GCE node
I20260807 19:44:04.833945 4292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:04.835158 4292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:04.836450 4292 hybrid_clock.cc:648] HybridClock initialized: now 1786131844836426 us; error 62 us; skew 500 ppm
I20260807 19:44:04.836853 4292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:04 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:04.840575 4292 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:44:04.842418 4292 webserver.cc:460] Webserver started at http://127.0.198.194:41521/ using document root <none> and password file <none>
I20260807 19:44:04.843223 4292 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:04.843422 4292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:04.843775 4292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:04.847030 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "84b40651e32344a9a81b57a86acdfda3"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "4f2d3d5193cde6fea1485a6ec42effc3"
server_key_iv: "eb9ffa36134f5a9e04ffdf4e0434ef3d"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.848021 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "84b40651e32344a9a81b57a86acdfda3"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "4f2d3d5193cde6fea1485a6ec42effc3"
server_key_iv: "eb9ffa36134f5a9e04ffdf4e0434ef3d"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.854094 4292 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:04.858289 4310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.859975 4292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:04.860237 4292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "84b40651e32344a9a81b57a86acdfda3"
format_stamp: "Formatted at 2026-08-07 19:44:04 on dist-test-slave-btw8"
server_key: "4f2d3d5193cde6fea1485a6ec42effc3"
server_key_iv: "eb9ffa36134f5a9e04ffdf4e0434ef3d"
server_key_version: "encryptionkey@0"
I20260807 19:44:04.860517 4292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:04.891052 4292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:04.896940 4292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:04.897264 4292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:04.898454 4292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:04.900192 4292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:04.900290 4292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.900453 4292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:04.900512 4292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:04.940486 4292 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:43277
I20260807 19:44:04.940526 4433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:43277 every 8 connection(s)
I20260807 19:44:04.941910 4292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:04.947587 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4292
I20260807 19:44:04.947849 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:04.948499 795 external_mini_cluster.cc:1428] Setting key 6507177bb9e7ccd48b627044ee04d5e9
Aug 07 19:44:04 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:04.972327 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:04.945140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:34595 (local address 127.0.198.254:39279)
0807 19:44:04.945751 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:04.945757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:04.946699 (+ 942us) server_negotiation.cc:407] Connection header received
0807 19:44:04.949668 (+ 2969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:04.949674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:04.949786 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:04.949934 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:04.952186 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.952951 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.956052 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.956549 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.956678 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:04.962288 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:04.962296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:04.962302 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:04.962361 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:04.967550 (+ 5189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:04.968824 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:04.968834 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:04.968841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:04.969084 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:04.969689 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:04.969697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:04.969703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:04.970153 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:04.970332 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:04.971426 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0807 19:44:04.971765 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":309,"threads_started":1}
I20260807 19:44:04.974380 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:04.945526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39279 (local address 127.0.198.194:34595)
0807 19:44:04.946436 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:04.946491 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:04.949297 (+ 2806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:04.950244 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:04.950254 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:04.950856 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:04.951904 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:04.951935 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.953165 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.953175 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:04.955802 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:04.955831 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:04.956860 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:04.956872 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:04.957290 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:04.958224 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:04.958266 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:04.961784 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:04.967815 (+ 6031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:04.967826 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:04.967845 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:04.968476 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:04.969260 (+ 784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:04.969265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:04.969268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:04.969530 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:04.970345 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:04.970354 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:04.970863 (+ 509us) client_negotiation.cc:769] Sending connection context
0807 19:44:04.971181 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:44:04.972033 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":332,"threads_started":1}
I20260807 19:44:04.977447 4434 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39279
I20260807 19:44:04.977945 4434 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
I20260807 19:44:04.978858 4434 heartbeater.cc:511] Master 127.0.198.254:39279 requested a full tablet report, sending...
I20260807 19:44:04.980989 4082 ts_manager.cc:194] Registered new tserver with Master: 84b40651e32344a9a81b57a86acdfda3 (127.0.198.194:43277)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
I20260807 19:44:04.982220 4082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:34595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:44:05.017001 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:05.386117 4440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:05.386478 4440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:05.386596 4440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:44:05.388895 4286 heartbeater.cc:503] Master 127.0.198.254:39279 was elected leader, sending a full tablet report...
W20260807 19:44:05.396025 4440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:05.396179 4440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:05.396385 4440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:05.407209 4440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:05.408946 4440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:05.410440 4440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:05.426586 4445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:05.426671 4446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:05.427012 4448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:05.427735 4440 server_base.cc:1034] running on GCE node
I20260807 19:44:05.428515 4440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:05.429670 4440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:05.430969 4440 hybrid_clock.cc:648] HybridClock initialized: now 1786131845430901 us; error 52 us; skew 500 ppm
I20260807 19:44:05.431514 4440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:05.435840 4440 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:44:05.437731 4440 webserver.cc:460] Webserver started at http://127.0.198.195:36779/ using document root <none> and password file <none>
I20260807 19:44:05.438581 4440 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:05.438768 4440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:05.439139 4440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:05.442652 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "d693831594f444219dd8f636e94e3bf5"
format_stamp: "Formatted at 2026-08-07 19:44:05 on dist-test-slave-btw8"
server_key: "ea55660a945b7b067ef707b4a21895ce"
server_key_iv: "52b5b87eb1fda432c34695d39b91a162"
server_key_version: "encryptionkey@0"
I20260807 19:44:05.443816 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "d693831594f444219dd8f636e94e3bf5"
format_stamp: "Formatted at 2026-08-07 19:44:05 on dist-test-slave-btw8"
server_key: "ea55660a945b7b067ef707b4a21895ce"
server_key_iv: "52b5b87eb1fda432c34695d39b91a162"
server_key_version: "encryptionkey@0"
I20260807 19:44:05.449954 4440 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:05.454591 4455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:05.456290 4440 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:05.456521 4440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "d693831594f444219dd8f636e94e3bf5"
format_stamp: "Formatted at 2026-08-07 19:44:05 on dist-test-slave-btw8"
server_key: "ea55660a945b7b067ef707b4a21895ce"
server_key_iv: "52b5b87eb1fda432c34695d39b91a162"
server_key_version: "encryptionkey@0"
I20260807 19:44:05.456779 4440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:05.471711 4440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:05.476941 4440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:05.477320 4440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:05.478459 4440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:05.480074 4440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:05.480202 4440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:05.480371 4440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:05.480463 4440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:05.522195 4440 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36909
I20260807 19:44:05.522233 4574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36909 every 8 connection(s)
I20260807 19:44:05.523895 4440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:05.525885 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4440
I20260807 19:44:05.526149 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:05.526846 795 external_mini_cluster.cc:1428] Setting key c07f4c20be71512c54dd2d9e8832bfe4
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:05.547325 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.526940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:48511 (local address 127.0.198.254:39279)
0807 19:44:05.527571 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:05.527577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:05.528881 (+ 1304us) server_negotiation.cc:407] Connection header received
0807 19:44:05.531169 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:05.531173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:05.531261 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:05.531424 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:05.533075 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.533883 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.535768 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.536210 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.536345 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.540374 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:05.540380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:05.540383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:05.540421 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:05.544175 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.544993 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.544998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.545000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.545128 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.545554 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.545558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.545560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.545860 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:05.545988 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:05.546822 (+ 834us) server_negotiation.cc:299] Negotiation successful
0807 19:44:05.547014 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":323,"threads_started":1}
I20260807 19:44:05.548457 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.527666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39279 (local address 127.0.198.195:48511)
0807 19:44:05.528684 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:05.528747 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:05.530893 (+ 2146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:05.531671 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:05.531680 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:05.532280 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:05.532903 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:05.532922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.534083 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.534088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:05.535615 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:05.535629 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.536464 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.536469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:05.536696 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.537442 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:05.537467 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:05.540067 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:05.544345 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:05.544354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:05.544370 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:05.544855 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:05.545230 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:05.545233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:05.545235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:05.545440 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:05.546019 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:05.546027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:05.546439 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 19:44:05.546689 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 19:44:05.547134 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":423,"threads_started":1}
I20260807 19:44:05.550194 4575 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39279
I20260807 19:44:05.550506 4575 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:05.551265 4575 heartbeater.cc:511] Master 127.0.198.254:39279 requested a full tablet report, sending...
I20260807 19:44:05.552955 4082 ts_manager.cc:194] Registered new tserver with Master: d693831594f444219dd8f636e94e3bf5 (127.0.198.195:36909)
I20260807 19:44:05.553867 4082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:48511
I20260807 19:44:05.556268 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:05.622422 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.605163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.0.198.254:39279)
0807 19:44:05.605360 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:05.605365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:05.605859 (+ 494us) server_negotiation.cc:407] Connection header received
0807 19:44:05.606012 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:05.606016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:05.606101 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:05.606215 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:05.607866 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.608682 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.610042 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.610471 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.610572 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.615481 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:05.615489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:05.615493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:05.615540 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:05.619396 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.620392 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.620397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.620400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.620551 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.621032 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.621035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.621038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.621422 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:05.621630 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:05.621957 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:44:05.622159 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 19:44:05.623118 4082 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51708
I20260807 19:44:05.625149 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4147
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:05.644423 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.626446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39044 (local address 127.0.198.193:41551)
0807 19:44:05.627437 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:05.627444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:05.627472 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:44:05.627538 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:05.627545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:05.627767 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:05.627895 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:05.629160 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.630234 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.631569 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.632190 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.632313 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.636815 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:05.636838 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:05.636855 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:05.636910 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:05.640506 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.641477 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.641487 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.641490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.641672 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.642216 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.642223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.642226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.642745 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:05.643109 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:05.643548 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 19:44:05.643921 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":373,"threads_started":1}
W20260807 19:44:05.646147 4280 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39044
W20260807 19:44:05.647629 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4147: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:44:05.647933 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4147
I20260807 19:44:05.667763 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4292
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:44:05.687284 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.668889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.0.198.194:43277)
0807 19:44:05.669584 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:05.669592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:05.669618 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:44:05.669679 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:05.669686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:05.669908 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:05.670107 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:05.671601 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.672628 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.673979 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.674575 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.674739 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.680057 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:05.680077 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:05.680095 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:05.680162 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:05.683710 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.684610 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.684627 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.684631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.684795 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.685278 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.685284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.685288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.685706 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:05.686081 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:05.686461 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:44:05.686827 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":431,"threads_started":1}
W20260807 19:44:05.688910 4429 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39524
W20260807 19:44:05.690236 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:44:05.690428 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4292
I20260807 19:44:05.709525 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4440
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131845, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:44:05.730014 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:05.710817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.0.198.195:36909)
0807 19:44:05.711695 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:05.711734 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:05.711786 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:44:05.711850 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:05.711857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:05.712226 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:05.712425 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:05.714215 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.715327 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.716863 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:05.717574 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:05.717763 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:05.722856 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:05.722875 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:05.722893 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:05.722940 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:05.726410 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.727554 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.727564 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.727567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.727778 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:05.728301 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:05.728306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:05.728308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:05.728740 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:05.729005 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:05.729467 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 19:44:05.729712 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":505,"threads_started":1}
W20260807 19:44:05.731540 4570 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38724
W20260807 19:44:05.732851 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4440: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:44:05.733063 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4440
I20260807 19:44:05.752601 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4046
I20260807 19:44:05.753981 4114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:05.863045 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4046
2026-08-07T19:44:05Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4192 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4594](info): set up 2 sockets
Aug 07 19:44:05 dist-test-slave-btw8 krb5kdc[4594](info): commencing operation
krb5kdc: starting...
W20260807 19:44:07.036397 795 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:07 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:07.323527 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:32917
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:32917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:07.702494 4609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:07.702821 4609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:07.702997 4609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:07.711210 4609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:07.711408 4609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:07.711499 4609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:07.711612 4609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:07.711694 4609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:07.722630 4609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:32917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:32917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:07.724221 4609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:07.725610 4609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:07.741564 4615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:07.741497 4617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:07.742347 4614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:07.742493 4609 server_base.cc:1034] running on GCE node
I20260807 19:44:07.743665 4609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:07.745240 4609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:07.746475 4609 hybrid_clock.cc:648] HybridClock initialized: now 1786131847746430 us; error 34 us; skew 500 ppm
I20260807 19:44:07.746833 4609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:07 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:07.754779 4609 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:07.756625 4609 webserver.cc:460] Webserver started at http://127.0.198.254:38711/ using document root <none> and password file <none>
I20260807 19:44:07.757360 4609 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:07.757477 4609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:07.757758 4609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:07.761161 4609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-btw8"
server_key: "d3e1873a5825b7d6dfc090489b7277cf"
server_key_iv: "1d4f22c90291528e9d1231cea910969a"
server_key_version: "encryptionkey@0"
I20260807 19:44:07.762265 4609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-btw8"
server_key: "d3e1873a5825b7d6dfc090489b7277cf"
server_key_iv: "1d4f22c90291528e9d1231cea910969a"
server_key_version: "encryptionkey@0"
I20260807 19:44:07.770769 4609 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.003s
I20260807 19:44:07.776053 4625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:07.778359 4609 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:44:07.778591 4609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-btw8"
server_key: "d3e1873a5825b7d6dfc090489b7277cf"
server_key_iv: "1d4f22c90291528e9d1231cea910969a"
server_key_version: "encryptionkey@0"
I20260807 19:44:07.778877 4609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:07.816567 4609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:07.821650 4609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:07.821964 4609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:07.843652 4609 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:32917
I20260807 19:44:07.843683 4678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:32917 every 8 connection(s)
I20260807 19:44:07.845237 4609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:07.849303 4679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:07.853607 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4609
I20260807 19:44:07.853860 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:07.854529 795 external_mini_cluster.cc:1428] Setting key f9cbad10720f9dfcf5eaba62b1585de5
I20260807 19:44:07.860803 4679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: Bootstrap starting.
I20260807 19:44:07.864880 4679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:07.866204 4679 log.cc:824] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:07.870616 4679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: No bootstrap required, opened a new log
Aug 07 19:44:07 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:07.879761 4679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } }
I20260807 19:44:07.880326 4679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:07.880453 4679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bcc01ec5d54cbe9d388ea0c7166ea5, State: Initialized, Role: FOLLOWER
I20260807 19:44:07.881219 4679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } }
I20260807 19:44:07.881523 4679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:07.881533 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:07.856750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.0.198.254:32917)
0807 19:44:07.857639 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:07.857651 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:07.857701 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:44:07.859633 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:07.859657 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:07.860076 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:07.860433 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:07.862551 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:07.864180 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:07.866066 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:07.866975 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:07.867211 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:07.872858 (+ 5647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:07.872869 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:07.872889 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:07.872933 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:07.876715 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:07.877534 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:07.877554 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:07.877562 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:07.877760 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:07.878222 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:07.878227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:07.878231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:07.878848 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:07.879175 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:07.879723 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 19:44:07.880229 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":415,"threads_started":1}
I20260807 19:44:07.881726 4679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:07.881958 4679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:07.885553 4679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } }
I20260807 19:44:07.886058 4679 leader_election.cc:302] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17bcc01ec5d54cbe9d388ea0c7166ea5; no voters:
I20260807 19:44:07.886819 4679 leader_election.cc:288] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:07.886965 4685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:07.889456 4679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:07.894224 4685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [term 1 LEADER]: Becoming Leader. State: Replica: 17bcc01ec5d54cbe9d388ea0c7166ea5, State: Running, Role: LEADER
I20260807 19:44:07.895062 4685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } }
I20260807 19:44:07.898824 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } } }
I20260807 19:44:07.899421 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:07.898869 4687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17bcc01ec5d54cbe9d388ea0c7166ea5. Latest consensus state: current_term: 1 leader_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bcc01ec5d54cbe9d388ea0c7166ea5" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 32917 } } }
I20260807 19:44:07.905435 4687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5 [sys.catalog]: This master's current role is: LEADER
W20260807 19:44:07.905632 4702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:44:07.905799 4702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:44:07.906527 4705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:07.910270 4705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:07.921108 4705 catalog_manager.cc:1347] Generated new cluster ID: 25d4f28412d4483ba450016d9b317d0b
I20260807 19:44:07.921285 4705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:07.954439 4705 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:07.955967 4705 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:07.966275 4705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17bcc01ec5d54cbe9d388ea0c7166ea5: Generated new TSK 0
I20260807 19:44:07.967273 4705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:44:08.041206 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:08.410125 4709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:08.410511 4709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:08.410706 4709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:08.419749 4709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:08.419910 4709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:08.420115 4709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:08.432067 4709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:08.433672 4709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:08.435196 4709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:08.451992 4714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:08.451968 4715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:08.451946 4717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:08.452865 4709 server_base.cc:1034] running on GCE node
I20260807 19:44:08.453533 4709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:08.454829 4709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:08.456207 4709 hybrid_clock.cc:648] HybridClock initialized: now 1786131848456121 us; error 67 us; skew 500 ppm
I20260807 19:44:08.456681 4709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:08 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:08.460675 4709 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:44:08.462587 4709 webserver.cc:460] Webserver started at http://127.0.198.193:41263/ using document root <none> and password file <none>
I20260807 19:44:08.463513 4709 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:08.463685 4709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:08.464071 4709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:08.467886 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "e4f0cad6fca647cf80a33b656d80083b"
format_stamp: "Formatted at 2026-08-07 19:44:08 on dist-test-slave-btw8"
server_key: "7d5bd573d5239be207577024cdd4c1da"
server_key_iv: "f5c83c8d5d8ebc97af6092477ccb6dfe"
server_key_version: "encryptionkey@0"
I20260807 19:44:08.468911 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "e4f0cad6fca647cf80a33b656d80083b"
format_stamp: "Formatted at 2026-08-07 19:44:08 on dist-test-slave-btw8"
server_key: "7d5bd573d5239be207577024cdd4c1da"
server_key_iv: "f5c83c8d5d8ebc97af6092477ccb6dfe"
server_key_version: "encryptionkey@0"
I20260807 19:44:08.475659 4709 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:44:08.480104 4724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:08.481951 4709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:44:08.482203 4709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "e4f0cad6fca647cf80a33b656d80083b"
format_stamp: "Formatted at 2026-08-07 19:44:08 on dist-test-slave-btw8"
server_key: "7d5bd573d5239be207577024cdd4c1da"
server_key_iv: "f5c83c8d5d8ebc97af6092477ccb6dfe"
server_key_version: "encryptionkey@0"
I20260807 19:44:08.482450 4709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:08.503234 4709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:08.508427 4709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:08.508785 4709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:08.509919 4709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:08.511574 4709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:08.511660 4709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:08.511808 4709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:08.511869 4709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:08.551213 4709 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:41119
I20260807 19:44:08.551261 4843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:41119 every 8 connection(s)
I20260807 19:44:08.552939 4709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:08.561019 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4709
I20260807 19:44:08.561326 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:08.562021 795 external_mini_cluster.cc:1428] Setting key 5771ff59ff09b1c82d7d5a0ee7feebf0
Aug 07 19:44:08 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:08.575488 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:08.556103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40985 (local address 127.0.198.254:32917)
0807 19:44:08.556639 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:08.556644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:08.557620 (+ 976us) server_negotiation.cc:407] Connection header received
0807 19:44:08.559723 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:08.559729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:08.559846 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:08.559983 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:08.561890 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:08.562691 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:08.564472 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:08.564878 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:08.565027 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:08.569206 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:08.569211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:08.569215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:08.569254 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:08.572146 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:08.572981 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:08.572991 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:08.572993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:08.573112 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:08.573590 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:08.573600 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:08.573603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:08.573895 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:08.574059 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:08.574837 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 19:44:08.575079 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":312,"threads_started":1}
I20260807 19:44:08.576594 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:08.556555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:32917 (local address 127.0.198.193:40985)
0807 19:44:08.557395 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:08.557439 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:08.559323 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:08.560260 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:08.560270 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:08.560879 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:08.561673 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:08.561695 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:08.562888 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:08.562893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:08.564281 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:08.564295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:08.565137 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:08.565145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:08.565402 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:08.566203 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:08.566230 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:08.568919 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:08.572328 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:08.572339 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:08.572360 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:08.572834 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:08.573244 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:08.573248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:08.573252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:08.573459 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:08.574043 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:08.574051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:08.574428 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 19:44:08.574715 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 19:44:08.575303 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":326,"threads_started":1}
I20260807 19:44:08.578336 4844 heartbeater.cc:347] Connected to a master server at 127.0.198.254:32917
I20260807 19:44:08.578678 4844 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:08.579541 4844 heartbeater.cc:511] Master 127.0.198.254:32917 requested a full tablet report, sending...
I20260807 19:44:08.581725 4643 ts_manager.cc:194] Registered new tserver with Master: e4f0cad6fca647cf80a33b656d80083b (127.0.198.193:41119)
I20260807 19:44:08.583185 4643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:40985
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:44:08.623513 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:08.987699 4851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:08.988121 4851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:08.988318 4851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:08.997360 4851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:08.997563 4851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:08.997772 4851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:09.008742 4851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:09.010286 4851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:09.011914 4851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:09.028111 4857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.028100 4859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.028385 4856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:09.028829 4851 server_base.cc:1034] running on GCE node
I20260807 19:44:09.029697 4851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:09.030750 4851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:09.031999 4851 hybrid_clock.cc:648] HybridClock initialized: now 1786131849031953 us; error 38 us; skew 500 ppm
I20260807 19:44:09.032382 4851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:09.036270 4851 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:44:09.038086 4851 webserver.cc:460] Webserver started at http://127.0.198.194:46797/ using document root <none> and password file <none>
I20260807 19:44:09.038944 4851 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:09.039124 4851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:09.039567 4851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:09.043004 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "9f9bd70ed8dc4a56b3939a79febe0dbe"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "a32e7d59bc6776a42f10c8f648e43967"
server_key_iv: "8235b530c45d2419306ec5bf2a0e79e6"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.043982 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "9f9bd70ed8dc4a56b3939a79febe0dbe"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "a32e7d59bc6776a42f10c8f648e43967"
server_key_iv: "8235b530c45d2419306ec5bf2a0e79e6"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.050302 4851 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:09.054613 4866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.056401 4851 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:09.056644 4851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "9f9bd70ed8dc4a56b3939a79febe0dbe"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "a32e7d59bc6776a42f10c8f648e43967"
server_key_iv: "8235b530c45d2419306ec5bf2a0e79e6"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.056921 4851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:09.083630 4851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:09.089529 4851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:09.089833 4851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:09.091054 4851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:09.092679 4851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:09.092777 4851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.092875 4851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:09.092923 4851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.135151 4851 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35563
I20260807 19:44:09.135217 4985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35563 every 8 connection(s)
I20260807 19:44:09.138141 4851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:09.142795 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4851
I20260807 19:44:09.143085 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:09.143797 795 external_mini_cluster.cc:1428] Setting key 89045773964d5c8e053ae2dc62ce134d
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:09.167002 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.141504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:49011 (local address 127.0.198.254:32917)
0807 19:44:09.142048 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.142053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.143400 (+ 1347us) server_negotiation.cc:407] Connection header received
0807 19:44:09.145739 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.145744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.145882 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.146005 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.148363 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.149632 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.151768 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.152578 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.152752 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.158013 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.158020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.158025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.158072 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.162137 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.163756 (+ 1619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.163762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.163764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.163926 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.164645 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.164649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.164651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.165025 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.165266 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.166396 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.166688 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":291,"threads_started":1}
I20260807 19:44:09.168836 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.142141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:32917 (local address 127.0.198.194:49011)
0807 19:44:09.143134 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:09.143201 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:09.145311 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:09.146391 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:09.146404 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:09.147321 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:09.148111 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.148133 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.149875 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.149880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.151555 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.151572 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.152897 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.152904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.153216 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.154237 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:09.154265 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:09.157499 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:09.162346 (+ 4847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.162363 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.162389 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.163212 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.164086 (+ 874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.164095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.164101 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.164455 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.165248 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:09.165260 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:09.165870 (+ 610us) client_negotiation.cc:769] Sending connection context
0807 19:44:09.166251 (+ 381us) client_negotiation.cc:240] Negotiation successful
0807 19:44:09.166937 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":437,"threads_started":1}
I20260807 19:44:09.170835 4986 heartbeater.cc:347] Connected to a master server at 127.0.198.254:32917
I20260807 19:44:09.171191 4986 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:09.172060 4986 heartbeater.cc:511] Master 127.0.198.254:32917 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
I20260807 19:44:09.173840 4643 ts_manager.cc:194] Registered new tserver with Master: 9f9bd70ed8dc4a56b3939a79febe0dbe (127.0.198.194:35563)
I20260807 19:44:09.174830 4643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:49011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:44:09.211931 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:09.586660 4844 heartbeater.cc:503] Master 127.0.198.254:32917 was elected leader, sending a full tablet report...
W20260807 19:44:09.588838 4992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:09.589243 4992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:09.589367 4992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:09.597435 4992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:09.597568 4992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:09.597761 4992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:09.608112 4992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:32917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:09.609601 4992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:09.611038 4992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:09.625922 4997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.625890 5000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.625972 4998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:09.627625 4992 server_base.cc:1034] running on GCE node
I20260807 19:44:09.628367 4992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:09.629611 4992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:09.630903 4992 hybrid_clock.cc:648] HybridClock initialized: now 1786131849630860 us; error 78 us; skew 500 ppm
I20260807 19:44:09.631326 4992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:09.635273 4992 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:44:09.637077 4992 webserver.cc:460] Webserver started at http://127.0.198.195:46249/ using document root <none> and password file <none>
I20260807 19:44:09.637944 4992 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:09.638126 4992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:09.638484 4992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:09.642349 4992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "82837816e98d4e41a60b17df02fffde8"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "82c60e28d2e969a9b8045c5df8a6421f"
server_key_iv: "71dc1272afcbae50350675ea54495fc2"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.643289 4992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "82837816e98d4e41a60b17df02fffde8"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "82c60e28d2e969a9b8045c5df8a6421f"
server_key_iv: "71dc1272afcbae50350675ea54495fc2"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.649466 4992 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:09.653977 5007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.655766 4992 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:09.655992 4992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "82837816e98d4e41a60b17df02fffde8"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-btw8"
server_key: "82c60e28d2e969a9b8045c5df8a6421f"
server_key_iv: "71dc1272afcbae50350675ea54495fc2"
server_key_version: "encryptionkey@0"
I20260807 19:44:09.656252 4992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:09.675868 4992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:09.681630 4992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:09.681989 4992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:09.683146 4992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:09.685371 4992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:09.685508 4992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.685696 4992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:09.685782 4992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.728310 4992 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:38753
I20260807 19:44:09.728376 5127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:38753 every 8 connection(s)
I20260807 19:44:09.729773 4992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:09.733186 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 4992
I20260807 19:44:09.733404 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:09.734032 795 external_mini_cluster.cc:1428] Setting key a8ec2402f8c34383922e7677d28c6835
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:09.755576 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.732836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:56351 (local address 127.0.198.254:32917)
0807 19:44:09.733438 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.733445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.734781 (+ 1336us) server_negotiation.cc:407] Connection header received
0807 19:44:09.737673 (+ 2892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.737682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.737799 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.737905 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.739873 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.740726 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.742710 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.743144 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.743280 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.747919 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.747926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.747930 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.747981 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.751998 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.752991 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.752995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.752998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.753126 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.753602 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.753605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.753607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.753923 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.754051 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.754967 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.755188 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":318,"threads_started":1}
I20260807 19:44:09.756739 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.733520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:32917 (local address 127.0.198.195:56351)
0807 19:44:09.734522 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:09.734566 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:09.737300 (+ 2734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:09.738238 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:09.738248 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:09.738865 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:09.739659 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.739683 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.740918 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.740922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.742449 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.742466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.743544 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.743552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.743862 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.744862 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:09.744907 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:09.747609 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:09.752212 (+ 4603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.752221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.752257 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.752848 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.753258 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.753262 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.753265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.753489 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.754117 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:09.754124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:09.754522 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 19:44:09.754826 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 19:44:09.755304 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":378,"threads_started":1}
I20260807 19:44:09.758462 5128 heartbeater.cc:347] Connected to a master server at 127.0.198.254:32917
I20260807 19:44:09.758853 5128 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:09.759853 5128 heartbeater.cc:511] Master 127.0.198.254:32917 requested a full tablet report, sending...
I20260807 19:44:09.761811 4643 ts_manager.cc:194] Registered new tserver with Master: 82837816e98d4e41a60b17df02fffde8 (127.0.198.195:38753)
I20260807 19:44:09.763393 4643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:56351
I20260807 19:44:09.764739 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:09.833137 5138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.814161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.0.198.193:41119)
0807 19:44:09.815071 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.815082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.815124 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:44:09.815214 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.815230 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.815552 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.815757 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.817477 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.818682 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.820375 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.821227 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.821402 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.826251 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.826273 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.826290 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.826349 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.829717 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.830614 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.830623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.830625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.830784 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.831218 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.831224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.831232 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.831718 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.832066 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.832417 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.832734 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":460,"threads_started":1}
Aug 07 19:44:09 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:09.858055 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.839021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.0.198.254:32917)
0807 19:44:09.839286 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.839294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.839616 (+ 322us) server_negotiation.cc:407] Connection header received
0807 19:44:09.839735 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.839740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.839857 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.839956 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.841468 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.842242 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.844253 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.844950 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.845101 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.850602 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.850610 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.850616 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.850670 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.854328 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.855261 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.855269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.855273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.855558 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.856129 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.856134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.856136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.856519 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.856820 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.857277 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.857616 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260807 19:44:09.858726 4643 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45160
I20260807 19:44:09.860335 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4709
I20260807 19:44:09.877172 5138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.861587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57458 (local address 127.0.198.193:41119)
0807 19:44:09.861838 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.861844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.862137 (+ 293us) server_negotiation.cc:407] Connection header received
0807 19:44:09.862374 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.862378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.862447 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.862559 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.864101 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.864910 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.866330 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.866832 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.867010 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.870868 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.870875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.870878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.870925 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.874033 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.874944 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.874950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.874953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.875115 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.875643 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.875648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.875651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.875934 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.876219 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.876570 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.876816 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:44:09.877847 4839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:09.986981 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4709
I20260807 19:44:10.007336 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4851
Aug 07 19:44:10 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:44:10.028791 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:10.008421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.0.198.194:35563)
0807 19:44:10.009032 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:10.009040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:10.009073 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:44:10.009137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:10.009143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:10.009359 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:10.009500 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:10.011240 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.012492 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.014049 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.014712 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.014829 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:10.020594 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:10.020613 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:10.020631 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:10.020685 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:10.024835 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.025888 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.025897 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.025899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.026085 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.026654 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.026660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.026663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.027078 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:10.027458 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:10.027988 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 19:44:10.028260 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":311,"threads_started":1}
I20260807 19:44:10.030426 4981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:10.119117 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4851
I20260807 19:44:10.138782 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4992
Aug 07 19:44:10 dist-test-slave-btw8 krb5kdc[4594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:44:10.160233 5145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:10.140357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.0.198.195:38753)
0807 19:44:10.141167 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:10.141175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:10.141209 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:44:10.141280 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:10.141287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:10.141507 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:10.141657 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:10.143102 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.144274 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.146294 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.146955 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.147083 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:10.152967 (+ 5884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:10.152985 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:10.153008 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:10.153055 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:10.156528 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.157584 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.157597 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.157599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.157784 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.158309 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.158314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.158316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.158767 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:10.159048 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:10.159497 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 19:44:10.159904 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":472,"threads_started":1}
I20260807 19:44:10.161773 5123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:10.252259 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4992
I20260807 19:44:10.272938 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 4609
I20260807 19:44:10.274405 4674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:10.388329 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 4609
2026-08-07T19:44:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4523 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:10 dist-test-slave-btw8 krb5kdc[5150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:10 dist-test-slave-btw8 krb5kdc[5150](info): set up 2 sockets
Aug 07 19:44:10 dist-test-slave-btw8 krb5kdc[5150](info): commencing operation
krb5kdc: starting...
W20260807 19:44:11.569239 795 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:11 dist-test-slave-btw8 krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:11.868990 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37611
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:37611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:12.258332 5164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:12.258848 5164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:12.259079 5164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:12.267321 5164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:12.267537 5164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:12.267642 5164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:12.267761 5164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:12.267858 5164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:12.279637 5164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:37611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:12.281114 5164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:12.282542 5164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:12.295960 5173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:12.296051 5170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:12.296068 5171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:12.297219 5164 server_base.cc:1034] running on GCE node
I20260807 19:44:12.298241 5164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:12.300040 5164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:12.301333 5164 hybrid_clock.cc:648] HybridClock initialized: now 1786131852301292 us; error 41 us; skew 500 ppm
I20260807 19:44:12.301795 5164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:12 dist-test-slave-btw8 krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:12.305807 5164 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:12.307279 5164 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:12.307461 5164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:12.307857 5164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:12.311254 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "93aeefe43f0d4a0ba87d11f387f7babb"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "8b36ae6f055b159b96244f40c64b1904"
server_key_iv: "1c8b967b487919c6c0323d270e36169d"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.312255 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "93aeefe43f0d4a0ba87d11f387f7babb"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "8b36ae6f055b159b96244f40c64b1904"
server_key_iv: "1c8b967b487919c6c0323d270e36169d"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.318667 5164 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:44:12.323159 5180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:12.324990 5164 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:12.325289 5164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "93aeefe43f0d4a0ba87d11f387f7babb"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "8b36ae6f055b159b96244f40c64b1904"
server_key_iv: "1c8b967b487919c6c0323d270e36169d"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.325587 5164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:12.351559 5164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:12.356917 5164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:12.357184 5164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:12.380250 5164 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:37611
I20260807 19:44:12.380278 5241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:37611 every 8 connection(s)
I20260807 19:44:12.381311 5164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:12.385811 5242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:12.389133 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5164
I20260807 19:44:12.389393 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:12.390085 795 external_mini_cluster.cc:1428] Setting key a11c84452f713fb1bc0e656aec61332e
I20260807 19:44:12.401126 5242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: Bootstrap starting.
I20260807 19:44:12.406399 5242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:12.408097 5242 log.cc:824] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:44:12 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:12.411545 5242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: No bootstrap required, opened a new log
I20260807 19:44:12.417580 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:12.392391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.0.198.254:37611)
0807 19:44:12.393302 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:12.393315 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:12.393361 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:44:12.395100 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:12.395124 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:12.395649 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:12.396002 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:12.398178 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:12.400775 (+ 2597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:12.402973 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:12.403839 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:12.403999 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:12.409631 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:12.409640 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:12.409658 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:12.409701 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:12.412974 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:12.413823 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:12.413832 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:12.413838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:12.414040 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:12.414454 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:12.414461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:12.414469 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:12.415086 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:12.415466 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:12.415926 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 19:44:12.416437 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":439,"threads_started":1}
I20260807 19:44:12.420804 5242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } }
I20260807 19:44:12.421316 5242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:12.421450 5242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93aeefe43f0d4a0ba87d11f387f7babb, State: Initialized, Role: FOLLOWER
I20260807 19:44:12.422073 5242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } }
I20260807 19:44:12.422333 5242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:12.422479 5242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:12.422685 5242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:12.425416 5242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } }
I20260807 19:44:12.426044 5242 leader_election.cc:302] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93aeefe43f0d4a0ba87d11f387f7babb; no voters:
I20260807 19:44:12.426851 5242 leader_election.cc:288] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:12.427021 5247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:12.429838 5242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:12.430323 5247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [term 1 LEADER]: Becoming Leader. State: Replica: 93aeefe43f0d4a0ba87d11f387f7babb, State: Running, Role: LEADER
I20260807 19:44:12.431048 5247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } }
I20260807 19:44:12.434660 5249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93aeefe43f0d4a0ba87d11f387f7babb. Latest consensus state: current_term: 1 leader_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } } }
I20260807 19:44:12.434754 5248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aeefe43f0d4a0ba87d11f387f7babb" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37611 } } }
I20260807 19:44:12.435153 5249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:12.435257 5248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:12.436390 5255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:12.443747 5255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:12.455191 5255 catalog_manager.cc:1347] Generated new cluster ID: 40616824836f46e9970ccc1ed3649ced
I20260807 19:44:12.455405 5255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:12.468382 5255 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:12.470024 5255 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:12.476591 5255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: Generated new TSK 0
I20260807 19:44:12.477617 5255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:44:12.543573 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:12.922749 5269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:12.923091 5269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:12.923277 5269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:12.931823 5269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:12.931993 5269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:12.932209 5269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:12.943647 5269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:12.945148 5269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:12.946661 5269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:12.961875 5277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:12.961875 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:12.962133 5274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:12.962997 5269 server_base.cc:1034] running on GCE node
I20260807 19:44:12.963775 5269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:12.964932 5269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:12.966221 5269 hybrid_clock.cc:648] HybridClock initialized: now 1786131852966149 us; error 70 us; skew 500 ppm
I20260807 19:44:12.966665 5269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:12 dist-test-slave-btw8 krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:12.975219 5269 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:44:12.976732 5269 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:12.976903 5269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:12.977231 5269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:12.981302 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "c6fc64b596e047359f929ac59f63134a"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "aaf3edad5bb61e1a32ee6087858634fa"
server_key_iv: "72343e7aed53bb2d0a6383da8f53f0a6"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.982406 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "c6fc64b596e047359f929ac59f63134a"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "aaf3edad5bb61e1a32ee6087858634fa"
server_key_iv: "72343e7aed53bb2d0a6383da8f53f0a6"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.989030 5269 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:44:12.993058 5283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:12.994736 5269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:44:12.994895 5269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "c6fc64b596e047359f929ac59f63134a"
format_stamp: "Formatted at 2026-08-07 19:44:12 on dist-test-slave-btw8"
server_key: "aaf3edad5bb61e1a32ee6087858634fa"
server_key_iv: "72343e7aed53bb2d0a6383da8f53f0a6"
server_key_version: "encryptionkey@0"
I20260807 19:44:12.995111 5269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:13.015965 5269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:13.020975 5269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:13.021246 5269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:13.022333 5269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:13.023866 5269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:13.023963 5269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:13.024046 5269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:13.024112 5269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:13.061794 5269 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:45203
I20260807 19:44:13.061843 5401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:45203 every 8 connection(s)
I20260807 19:44:13.062811 5269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:13.063549 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5269
I20260807 19:44:13.063848 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:13.064646 795 external_mini_cluster.cc:1428] Setting key 80d9c787719c343018c44aadafac1ed0
Aug 07 19:44:13 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:13.087206 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:13.066801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:36907 (local address 127.0.198.254:37611)
0807 19:44:13.067311 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:13.067318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:13.067340 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:44:13.069350 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:13.069359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:13.069467 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:13.069605 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:13.071458 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.072807 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.074561 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.075408 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.075566 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:13.080298 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:13.080304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:13.080307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:13.080350 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:13.083333 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:13.084404 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:13.084410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:13.084414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:13.084570 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:13.085091 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:13.085096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:13.085098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:13.085432 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:13.085672 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:13.086512 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 19:44:13.086787 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":960,"threads_started":1}
I20260807 19:44:13.088341 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:13.066329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37611 (local address 127.0.198.193:36907)
0807 19:44:13.067181 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:13.067237 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:13.069041 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:13.069909 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:13.069919 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:13.070559 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:13.071210 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:13.071229 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.073028 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.073032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:13.074366 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:13.074381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.075723 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.075729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:13.075962 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:13.076746 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:13.076784 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:13.079978 (+ 3194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:13.083641 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:13.083654 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:13.083680 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:13.084240 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:13.084715 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:13.084721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:13.084725 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:13.084940 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:13.085641 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:13.085651 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:13.086074 (+ 423us) client_negotiation.cc:769] Sending connection context
0807 19:44:13.086404 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 19:44:13.087037 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":534,"threads_started":1}
I20260807 19:44:13.090329 5402 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37611
I20260807 19:44:13.090706 5402 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:44:13.091567 5402 heartbeater.cc:511] Master 127.0.198.254:37611 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:44:13.093880 5203 ts_manager.cc:194] Registered new tserver with Master: c6fc64b596e047359f929ac59f63134a (127.0.198.193:45203)
I20260807 19:44:13.095599 5203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:36907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:44:13.128288 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:13.502261 5408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:13.502595 5408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:13.502707 5408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:13.510514 5408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:13.510665 5408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:13.510798 5408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:13.521494 5408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:13.522920 5408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:13.524349 5408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:13.539587 5413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:13.539490 5414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:13.539520 5416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:13.540391 5408 server_base.cc:1034] running on GCE node
I20260807 19:44:13.541150 5408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:13.542284 5408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:13.543530 5408 hybrid_clock.cc:648] HybridClock initialized: now 1786131853543470 us; error 69 us; skew 500 ppm
I20260807 19:44:13.543900 5408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:13 dist-test-slave-btw8 krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:13.547688 5408 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:44:13.549110 5408 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:13.549265 5408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:13.549636 5408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:13.553159 5408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "d9819de3042448c8b9f8465208881742"
format_stamp: "Formatted at 2026-08-07 19:44:13 on dist-test-slave-btw8"
server_key: "b64f8423ebd4ab096b177412bacb9991"
server_key_iv: "d01f0b5cb4c80c57746c7384eff8d2ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:13.554141 5408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "d9819de3042448c8b9f8465208881742"
format_stamp: "Formatted at 2026-08-07 19:44:13 on dist-test-slave-btw8"
server_key: "b64f8423ebd4ab096b177412bacb9991"
server_key_iv: "d01f0b5cb4c80c57746c7384eff8d2ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:13.560489 5408 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 19:44:13.564759 5422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:13.566329 5408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:44:13.566572 5408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "d9819de3042448c8b9f8465208881742"
format_stamp: "Formatted at 2026-08-07 19:44:13 on dist-test-slave-btw8"
server_key: "b64f8423ebd4ab096b177412bacb9991"
server_key_iv: "d01f0b5cb4c80c57746c7384eff8d2ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:13.566844 5408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:13.589752 5408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:13.594373 5408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:13.594702 5408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:13.595825 5408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:13.597383 5408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:13.597509 5408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:13.597656 5408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:13.597745 5408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:13.637220 5408 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:46499
I20260807 19:44:13.637274 5540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:46499 every 8 connection(s)
I20260807 19:44:13.638273 5408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:13.648396 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5408
I20260807 19:44:13.648623 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:13.649142 795 external_mini_cluster.cc:1428] Setting key 9c65ae09c1fe8123413d5e3890e1b3bb
Aug 07 19:44:13 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:13.661677 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:13.641253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:55919 (local address 127.0.198.254:37611)
0807 19:44:13.641766 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:13.641772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:13.642829 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 19:44:13.644896 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:13.644902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:13.645010 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:13.645145 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:13.646888 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.647774 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.649689 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.650224 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.650369 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:13.654575 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:13.654581 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:13.654584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:13.654622 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:13.657792 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:13.658769 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:13.658778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:13.658788 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:13.658945 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:13.659558 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:13.659564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:13.659568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:13.659889 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:13.660113 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:13.660918 (+ 805us) server_negotiation.cc:299] Negotiation successful
0807 19:44:13.661208 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":258,"threads_started":1}
I20260807 19:44:13.662824 5544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:13.641820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37611 (local address 127.0.198.194:55919)
0807 19:44:13.642607 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:13.642649 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:13.644559 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:13.645408 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:13.645416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:13.646004 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:13.646664 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:13.646692 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.647964 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.647968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:13.649510 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:13.649524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:13.650467 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:13.650473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:13.650698 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:13.651650 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:13.651707 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:13.654294 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:13.658004 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:13.658020 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:13.658037 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:13.658605 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:13.659084 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:13.659093 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:13.659097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:13.659339 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:13.660050 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:13.660060 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:13.660479 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 19:44:13.660799 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 19:44:13.661443 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":331,"threads_started":1}
I20260807 19:44:13.665004 5541 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37611
I20260807 19:44:13.665385 5541 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:13.666142 5541 heartbeater.cc:511] Master 127.0.198.254:37611 requested a full tablet report, sending...
I20260807 19:44:13.667939 5203 ts_manager.cc:194] Registered new tserver with Master: d9819de3042448c8b9f8465208881742 (127.0.198.194:46499)
I20260807 19:44:13.669268 5203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:55919
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:44:13.714798 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:14.073733 5547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:14.074102 5547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:14.074280 5547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:14.082204 5547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:14.082398 5547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:14.082584 5547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:14.093530 5547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:14.094980 5547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:14.096382 5547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:44:14.098822 5402 heartbeater.cc:503] Master 127.0.198.254:37611 was elected leader, sending a full tablet report...
W20260807 19:44:14.111071 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:14.111096 5555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:14.111562 5553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:14.112722 5547 server_base.cc:1034] running on GCE node
I20260807 19:44:14.113575 5547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:14.114682 5547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:14.115913 5547 hybrid_clock.cc:648] HybridClock initialized: now 1786131854115874 us; error 32 us; skew 500 ppm
I20260807 19:44:14.116309 5547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:14 dist-test-slave-btw8 krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:14.120227 5547 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:44:14.121649 5547 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:14.121791 5547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:14.122259 5547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:14.125895 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "0467cccd5e9946c78cf2c5328c4032bf"
format_stamp: "Formatted at 2026-08-07 19:44:14 on dist-test-slave-btw8"
server_key: "932b217ff6398ef5b9349528008b9f1f"
server_key_iv: "6c0395c3512ec3fb5b81beed14529a38"
server_key_version: "encryptionkey@0"
I20260807 19:44:14.126951 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "0467cccd5e9946c78cf2c5328c4032bf"
format_stamp: "Formatted at 2026-08-07 19:44:14 on dist-test-slave-btw8"
server_key: "932b217ff6398ef5b9349528008b9f1f"
server_key_iv: "6c0395c3512ec3fb5b81beed14529a38"
server_key_version: "encryptionkey@0"
I20260807 19:44:14.133502 5547 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:14.137974 5561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:14.139883 5547 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:14.140107 5547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "0467cccd5e9946c78cf2c5328c4032bf"
format_stamp: "Formatted at 2026-08-07 19:44:14 on dist-test-slave-btw8"
server_key: "932b217ff6398ef5b9349528008b9f1f"
server_key_iv: "6c0395c3512ec3fb5b81beed14529a38"
server_key_version: "encryptionkey@0"
I20260807 19:44:14.140465 5547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:14.157888 5547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:14.163483 5547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:14.163813 5547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:14.164912 5547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:14.166405 5547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:14.166494 5547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:14.166576 5547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:14.166635 5547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:14.203557 5547 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:37547
I20260807 19:44:14.203590 5676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:37547 every 8 connection(s)
I20260807 19:44:14.204545 5547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:14.214425 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5547
I20260807 19:44:14.214696 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:14.215256 795 external_mini_cluster.cc:1428] Setting key b9010b55dc13a4df931ebf022aa1b535
Aug 07 19:44:14 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:14.228243 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.207804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:43611 (local address 127.0.198.254:37611)
0807 19:44:14.208321 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.208326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.209343 (+ 1017us) server_negotiation.cc:407] Connection header received
0807 19:44:14.211312 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.211316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.211467 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.211572 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:14.213158 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.213998 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.216458 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.217065 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.217172 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.221554 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:14.221559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:14.221562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:14.221599 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:14.224843 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.225682 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.225686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.225689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.225833 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.226265 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.226269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.226271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.226556 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:14.226680 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.227586 (+ 906us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.227865 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":240,"threads_started":1}
I20260807 19:44:14.229053 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.208413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37611 (local address 127.0.198.195:43611)
0807 19:44:14.209154 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.209202 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.210973 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.211822 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.211831 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.212377 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:14.212979 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.212996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.214218 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.214225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.216261 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.216277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.217367 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.217376 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.217652 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.218435 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:14.218475 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:14.221295 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:14.225019 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:14.225027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:14.225042 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:14.225540 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:14.225944 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:14.225947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:14.225949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:14.226138 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:14.226694 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:14.226701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:14.227061 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.227329 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.227863 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":311,"threads_started":1}
I20260807 19:44:14.230955 5677 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37611
I20260807 19:44:14.231431 5677 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:14.232673 5677 heartbeater.cc:511] Master 127.0.198.254:37611 requested a full tablet report, sending...
I20260807 19:44:14.234411 5203 ts_manager.cc:194] Registered new tserver with Master: 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547)
I20260807 19:44:14.235411 5203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:43611
I20260807 19:44:14.245841 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:44:14.267030 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.251105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.0.198.254:37611)
0807 19:44:14.251302 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.251308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.251766 (+ 458us) server_negotiation.cc:407] Connection header received
0807 19:44:14.251989 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.251994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.252090 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.252211 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:14.253762 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.254710 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.256288 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.256877 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.257003 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.260376 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:14.260383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:14.260387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:14.260435 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:14.263708 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.264626 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.264632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.264634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.264811 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.265347 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.265351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.265354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.265795 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:14.266012 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.266485 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.266761 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:44:14.272737 5203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:14.281090 5203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:14.305030 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.294886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:46499 (local address 127.0.0.1:48782)
0807 19:44:14.295721 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.295767 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.295946 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.297037 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.297046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.297080 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.297534 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.297551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.299467 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.299475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.301976 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.301994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303801 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.303808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.304111 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.304189 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.304345 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.304557 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":411,"threads_started":1}
I20260807 19:44:14.305372 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.295127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.0.198.194:46499)
0807 19:44:14.295740 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.295751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.296097 (+ 346us) server_negotiation.cc:407] Connection header received
0807 19:44:14.296220 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.296229 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.296531 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.296760 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.297750 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.299249 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.302176 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303556 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.303725 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.304782 (+ 1057us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.304876 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.305068 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":735,"threads_started":1}
I20260807 19:44:14.309526 5472 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fd9cb610492abdcc7c428428f66b (DEFAULT_TABLE table=test-table [id=d3a9e996896c4222aa25fdff03432323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:14.310235 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.296201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45203 (local address 127.0.0.1:47144)
0807 19:44:14.296810 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.296831 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.296944 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.300743 (+ 3799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.300749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.300778 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.301499 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.301516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303806 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.303810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.306731 (+ 2921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.306753 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.309196 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.309202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.309410 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.309480 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.309608 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.309770 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":4,"thread_start_us":321,"threads_started":1}
I20260807 19:44:14.310377 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.296483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.0.198.193:45203)
0807 19:44:14.300230 (+ 3747us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.300236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.300259 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:14.300332 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.300337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.300495 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.301284 (+ 789us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.301715 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303583 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.307005 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.308886 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.309104 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.309421 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.309686 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.309912 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3586,"thread_start_us":3416,"threads_started":1}
I20260807 19:44:14.310856 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.296910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:37547 (local address 127.0.0.1:35554)
0807 19:44:14.297512 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.297530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.297643 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.300602 (+ 2959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.300610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.300642 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.301443 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.301457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303803 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.303810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.307109 (+ 3299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.307128 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.309421 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.309430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.309741 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.309788 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.309888 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.310053 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":74,"thread_start_us":1113,"threads_started":1}
I20260807 19:44:14.311738 5472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fd9cb610492abdcc7c428428f66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:14.312418 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.297600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.0.198.195:37547)
0807 19:44:14.299946 (+ 2346us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.299955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.299990 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:44:14.300102 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.300113 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.300379 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.300596 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.301666 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.303599 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.307369 (+ 3770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.309119 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.311315 (+ 2196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.311668 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.311760 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.311981 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2039,"thread_start_us":1843,"threads_started":1}
I20260807 19:44:14.314679 5335 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fd9cb610492abdcc7c428428f66b (DEFAULT_TABLE table=test-table [id=d3a9e996896c4222aa25fdff03432323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:14.316252 5335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fd9cb610492abdcc7c428428f66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:14.316130 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 3e91fd9cb610492abdcc7c428428f66b (DEFAULT_TABLE table=test-table [id=d3a9e996896c4222aa25fdff03432323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:14.317497 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e91fd9cb610492abdcc7c428428f66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:14.328801 5697 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Bootstrap starting.
I20260807 19:44:14.332628 5698 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Bootstrap starting.
I20260807 19:44:14.334625 5697 tablet_bootstrap.cc:654] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:14.336862 5697 log.cc:824] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:14.338295 5698 tablet_bootstrap.cc:654] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:14.340071 5698 log.cc:824] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:14.341112 5697 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: No bootstrap required, opened a new log
I20260807 19:44:14.341449 5697 ts_tablet_manager.cc:1397] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260807 19:44:14.343178 5698 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: No bootstrap required, opened a new log
I20260807 19:44:14.339021 5699 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Bootstrap starting.
I20260807 19:44:14.343569 5698 ts_tablet_manager.cc:1397] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 19:44:14.347805 5699 tablet_bootstrap.cc:654] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:14.349380 5699 log.cc:824] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:14.352813 5697 raft_consensus.cc:348] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.353325 5697 raft_consensus.cc:374] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:14.353421 5699 tablet_bootstrap.cc:492] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: No bootstrap required, opened a new log
I20260807 19:44:14.353480 5697 raft_consensus.cc:729] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9819de3042448c8b9f8465208881742, State: Initialized, Role: FOLLOWER
I20260807 19:44:14.353739 5699 ts_tablet_manager.cc:1397] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260807 19:44:14.354192 5697 consensus_queue.cc:260] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.354256 5698 raft_consensus.cc:348] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.354796 5698 raft_consensus.cc:374] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:14.354929 5698 raft_consensus.cc:729] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0467cccd5e9946c78cf2c5328c4032bf, State: Initialized, Role: FOLLOWER
I20260807 19:44:14.355711 5698 consensus_queue.cc:260] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.357264 5677 heartbeater.cc:503] Master 127.0.198.254:37611 was elected leader, sending a full tablet report...
I20260807 19:44:14.358402 5698 ts_tablet_manager.cc:1428] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 19:44:14.360240 5541 heartbeater.cc:503] Master 127.0.198.254:37611 was elected leader, sending a full tablet report...
I20260807 19:44:14.363139 5697 ts_tablet_manager.cc:1428] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Time spent starting tablet: real 0.022s user 0.010s sys 0.010s
I20260807 19:44:14.367241 5699 raft_consensus.cc:348] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.367805 5699 raft_consensus.cc:374] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:14.367925 5699 raft_consensus.cc:729] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fc64b596e047359f929ac59f63134a, State: Initialized, Role: FOLLOWER
I20260807 19:44:14.368558 5699 consensus_queue.cc:260] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.370523 5699 ts_tablet_manager.cc:1428] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
W20260807 19:44:14.390799 5542 tablet.cc:2369] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:44:14.456231 5678 tablet.cc:2369] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:14.463502 5705 raft_consensus.cc:482] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:14.463913 5705 raft_consensus.cc:504] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.465969 5705 leader_election.cc:288] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9819de3042448c8b9f8465208881742 (127.0.198.194:46499), 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547)
I20260807 19:44:14.469316 5703 raft_consensus.cc:482] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:14.469720 5703 raft_consensus.cc:504] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.472136 5703 leader_election.cc:288] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6fc64b596e047359f929ac59f63134a (127.0.198.193:45203), 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547)
I20260807 19:44:14.478518 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.467850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:53911 (local address 127.0.198.195:37547)
0807 19:44:14.468061 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.468069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.468421 (+ 352us) server_negotiation.cc:407] Connection header received
0807 19:44:14.468588 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.468594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.468678 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.468787 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.469613 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.470845 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.474232 (+ 3387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.476459 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.476687 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.476817 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.477794 (+ 977us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.478044 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:44:14.483839 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.467605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:37547 (local address 127.0.198.193:53911)
0807 19:44:14.468301 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.468316 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.468455 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.468809 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.468812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.468832 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.469371 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.469385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.471005 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.471013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.473924 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.473951 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.477145 (+ 3194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.477156 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.477476 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.477540 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.479846 (+ 2306us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.483018 (+ 3172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":157,"thread_start_us":351,"threads_started":1}
I20260807 19:44:14.485886 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.466704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:51705 (local address 127.0.198.194:46499)
0807 19:44:14.467007 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.467017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.467575 (+ 558us) server_negotiation.cc:407] Connection header received
0807 19:44:14.467920 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.467928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.468017 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.468088 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.469175 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.470574 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.480132 (+ 9558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.482347 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.482623 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.482764 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.485161 (+ 2397us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.485412 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":80}
I20260807 19:44:14.486696 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.472566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45203 (local address 127.0.198.194:36963)
0807 19:44:14.473535 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.473560 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.473791 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.474467 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.474476 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.474516 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.475172 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.475189 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.477665 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.477675 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.481257 (+ 3582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.481287 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.484459 (+ 3172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.484470 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.484801 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.484861 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.485943 (+ 1082us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.486292 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":606,"threads_started":1}
I20260807 19:44:14.485769 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.472785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:36963 (local address 127.0.198.193:45203)
0807 19:44:14.472998 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.473005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.473649 (+ 644us) server_negotiation.cc:407] Connection header received
0807 19:44:14.473989 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.473996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.474086 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.474182 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.475442 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.477318 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.481587 (+ 4269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.483730 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.483975 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.484102 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.485093 (+ 991us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.485347 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":60}
I20260807 19:44:14.487231 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.466551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:46499 (local address 127.0.198.193:51705)
0807 19:44:14.467412 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.467441 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.467645 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.468279 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.468282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.468307 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.468806 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.468821 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.470749 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.470755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.479849 (+ 9094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.479873 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.484441 (+ 4568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.484453 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.484824 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.484888 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.486421 (+ 1533us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.486675 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":191,"thread_start_us":489,"threads_started":1}
I20260807 19:44:14.488786 5356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "d9819de3042448c8b9f8465208881742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fc64b596e047359f929ac59f63134a" is_pre_election: true
I20260807 19:44:14.488828 5631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "c6fc64b596e047359f929ac59f63134a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0467cccd5e9946c78cf2c5328c4032bf" is_pre_election: true
I20260807 19:44:14.489328 5356 raft_consensus.cc:2454] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9819de3042448c8b9f8465208881742 in term 0.
I20260807 19:44:14.489154 5494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "c6fc64b596e047359f929ac59f63134a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9819de3042448c8b9f8465208881742" is_pre_election: true
I20260807 19:44:14.489496 5631 raft_consensus.cc:2454] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fc64b596e047359f929ac59f63134a in term 0.
I20260807 19:44:14.489650 5494 raft_consensus.cc:2454] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fc64b596e047359f929ac59f63134a in term 0.
I20260807 19:44:14.490423 5287 leader_election.cc:302] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6fc64b596e047359f929ac59f63134a, d9819de3042448c8b9f8465208881742; no voters:
I20260807 19:44:14.490931 5705 raft_consensus.cc:2781] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:14.491092 5705 raft_consensus.cc:482] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:14.491197 5705 raft_consensus.cc:3037] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:14.491587 5426 leader_election.cc:302] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6fc64b596e047359f929ac59f63134a, d9819de3042448c8b9f8465208881742; no voters:
I20260807 19:44:14.492061 5703 raft_consensus.cc:2781] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:14.492219 5703 raft_consensus.cc:482] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:14.492334 5703 raft_consensus.cc:3037] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:14.494693 5705 raft_consensus.cc:504] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.495916 5494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "c6fc64b596e047359f929ac59f63134a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9819de3042448c8b9f8465208881742"
I20260807 19:44:14.496215 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.473956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:38985 (local address 127.0.198.195:37547)
0807 19:44:14.479745 (+ 5789us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.479757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.487166 (+ 7409us) server_negotiation.cc:407] Connection header received
0807 19:44:14.488374 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.488383 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.488492 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.488575 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:14.489320 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.490661 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.493177 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.494999 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.495225 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.495322 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.495689 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.495844 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5490,"thread_start_us":12458,"threads_started":1}
I20260807 19:44:14.496315 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.473709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:37547 (local address 127.0.198.194:38985)
0807 19:44:14.487016 (+ 13307us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:14.487040 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:14.487255 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:14.488843 (+ 1588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:14.488851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:14.488893 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:14.489107 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.489121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.490894 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.490902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.492919 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:14.492940 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.495203 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.495210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:14.495484 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.495528 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:44:14.495664 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 19:44:14.495848 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13181,"thread_start_us":13676,"threads_started":1}
I20260807 19:44:14.496502 5631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "c6fc64b596e047359f929ac59f63134a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0467cccd5e9946c78cf2c5328c4032bf"
I20260807 19:44:14.496631 5705 leader_election.cc:288] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [CANDIDATE]: Term 1 election: Requested vote from peers d9819de3042448c8b9f8465208881742 (127.0.198.194:46499), 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547)
I20260807 19:44:14.496723 5631 raft_consensus.cc:3037] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:14.495669 5703 raft_consensus.cc:504] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.497182 5494 raft_consensus.cc:2379] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c6fc64b596e047359f929ac59f63134a in current term 1: Already voted for candidate d9819de3042448c8b9f8465208881742 in this term.
I20260807 19:44:14.497514 5703 leader_election.cc:288] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [CANDIDATE]: Term 1 election: Requested vote from peers c6fc64b596e047359f929ac59f63134a (127.0.198.193:45203), 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547)
I20260807 19:44:14.497414 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "d9819de3042448c8b9f8465208881742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0467cccd5e9946c78cf2c5328c4032bf" is_pre_election: true
I20260807 19:44:14.498234 5356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "d9819de3042448c8b9f8465208881742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fc64b596e047359f929ac59f63134a"
I20260807 19:44:14.498550 5356 raft_consensus.cc:2379] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9819de3042448c8b9f8465208881742 in current term 1: Already voted for candidate c6fc64b596e047359f929ac59f63134a in this term.
I20260807 19:44:14.499536 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e91fd9cb610492abdcc7c428428f66b" candidate_uuid: "d9819de3042448c8b9f8465208881742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0467cccd5e9946c78cf2c5328c4032bf"
I20260807 19:44:14.500039 5426 leader_election.cc:302] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9819de3042448c8b9f8465208881742; no voters: 0467cccd5e9946c78cf2c5328c4032bf, c6fc64b596e047359f929ac59f63134a
I20260807 19:44:14.500355 5631 raft_consensus.cc:2454] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6fc64b596e047359f929ac59f63134a in term 1.
I20260807 19:44:14.500420 5703 raft_consensus.cc:2726] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:44:14.500725 5287 leader_election.cc:302] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0467cccd5e9946c78cf2c5328c4032bf, c6fc64b596e047359f929ac59f63134a; no voters: d9819de3042448c8b9f8465208881742
I20260807 19:44:14.501065 5705 raft_consensus.cc:2781] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:14.501506 5705 raft_consensus.cc:686] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 LEADER]: Becoming Leader. State: Replica: c6fc64b596e047359f929ac59f63134a, State: Running, Role: LEADER
I20260807 19:44:14.502022 5705 consensus_queue.cc:237] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } }
I20260807 19:44:14.507323 5203 catalog_manager.cc:5495] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a reported cstate change: term changed from 0 to 1, leader changed from <none> to c6fc64b596e047359f929ac59f63134a (127.0.198.193). New cstate: current_term: 1 leader_uuid: "c6fc64b596e047359f929ac59f63134a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6fc64b596e047359f929ac59f63134a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:14.553803 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.547945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.0.198.193:45203)
0807 19:44:14.548178 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.548185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.548346 (+ 161us) server_negotiation.cc:407] Connection header received
0807 19:44:14.548611 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.548615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.548698 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.548890 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:14.549278 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.550064 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.551558 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.551980 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.552122 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.552636 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:14.553168 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:14.553343 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.553404 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.553491 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:44:14.563820 5631 raft_consensus.cc:1261] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 1 FOLLOWER]: Refusing update from remote peer c6fc64b596e047359f929ac59f63134a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:14.563828 5494 raft_consensus.cc:1261] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Refusing update from remote peer c6fc64b596e047359f929ac59f63134a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:14.564869 5711 consensus_queue.cc:1035] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0467cccd5e9946c78cf2c5328c4032bf" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:14.565328 5705 consensus_queue.cc:1035] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9819de3042448c8b9f8465208881742" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 46499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:44:14.565754 5403 tablet.cc:2369] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:14.582463 5718 mvcc.cc:205] Tried to move back new op lower bound from 7315996076280336384 to 7315996076044247040. Current Snapshot: MvccSnapshot[applied={T|T < 7315996076280336384}]
I20260807 19:44:14.597468 5203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:14.597762 5203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:14.601967 5203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93aeefe43f0d4a0ba87d11f387f7babb: Sending DeleteTablet for 3 replicas of tablet 3e91fd9cb610492abdcc7c428428f66b
I20260807 19:44:14.603432 5472 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fd9cb610492abdcc7c428428f66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:14 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:48782
I20260807 19:44:14.603554 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fd9cb610492abdcc7c428428f66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:14 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:35554
I20260807 19:44:14.604521 5727 tablet_replica.cc:330] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: stopping tablet replica
I20260807 19:44:14.604655 5335 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e91fd9cb610492abdcc7c428428f66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:14 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:47144
I20260807 19:44:14.605058 5727 raft_consensus.cc:2229] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:14.605590 5727 raft_consensus.cc:2258] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:14.606050 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5269
I20260807 19:44:14.606427 5726 tablet_replica.cc:330] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: stopping tablet replica
I20260807 19:44:14.606853 5726 raft_consensus.cc:2229] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:14.607388 5726 raft_consensus.cc:2258] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:14.609884 5726 ts_tablet_manager.cc:1905] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:14.610217 5727 ts_tablet_manager.cc:1905] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:14.617553 5726 ts_tablet_manager.cc:1918] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:14.617892 5726 log.cc:1196] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/3e91fd9cb610492abdcc7c428428f66b
I20260807 19:44:14.618610 5726 ts_tablet_manager.cc:1939] T 3e91fd9cb610492abdcc7c428428f66b P d9819de3042448c8b9f8465208881742: Deleting consensus metadata
I20260807 19:44:14.620874 5185 catalog_manager.cc:4841] TS d9819de3042448c8b9f8465208881742 (127.0.198.194:46499): tablet 3e91fd9cb610492abdcc7c428428f66b (table test-table [id=d3a9e996896c4222aa25fdff03432323]) successfully deleted
I20260807 19:44:14.624017 5728 tablet_replica.cc:330] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: stopping tablet replica
I20260807 19:44:14.624610 5728 raft_consensus.cc:2229] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:14.625432 5728 raft_consensus.cc:2258] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:14.626734 5727 ts_tablet_manager.cc:1918] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:14.627058 5727 log.cc:1196] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/3e91fd9cb610492abdcc7c428428f66b
I20260807 19:44:14.627844 5727 ts_tablet_manager.cc:1939] T 3e91fd9cb610492abdcc7c428428f66b P 0467cccd5e9946c78cf2c5328c4032bf: Deleting consensus metadata
I20260807 19:44:14.628337 5728 ts_tablet_manager.cc:1905] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:14.630163 5185 catalog_manager.cc:4841] TS 0467cccd5e9946c78cf2c5328c4032bf (127.0.198.195:37547): tablet 3e91fd9cb610492abdcc7c428428f66b (table test-table [id=d3a9e996896c4222aa25fdff03432323]) successfully deleted
Aug 07 19:44:14 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:14.638106 5728 ts_tablet_manager.cc:1918] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:14.638509 5728 log.cc:1196] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/3e91fd9cb610492abdcc7c428428f66b
I20260807 19:44:14.639281 5728 ts_tablet_manager.cc:1939] T 3e91fd9cb610492abdcc7c428428f66b P c6fc64b596e047359f929ac59f63134a: Deleting consensus metadata
I20260807 19:44:14.641727 5185 catalog_manager.cc:4841] TS c6fc64b596e047359f929ac59f63134a (127.0.198.193:45203): tablet 3e91fd9cb610492abdcc7c428428f66b (table test-table [id=d3a9e996896c4222aa25fdff03432323]) successfully deleted
I20260807 19:44:14.644428 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.608233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.0.198.193:45203)
0807 19:44:14.608523 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.608530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.619494 (+ 10964us) server_negotiation.cc:407] Connection header received
0807 19:44:14.620202 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.620213 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.620368 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.620472 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:14.622925 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.624273 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.626685 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.627545 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.627670 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.633959 (+ 6289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:14.633975 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:14.633997 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:14.634059 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:14.639489 (+ 5430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.640696 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.640707 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.640711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.640941 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.642137 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.642152 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.642155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.642595 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:14.643047 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.643768 (+ 721us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.644046 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:44:14.645262 5396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:14.751928 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5269
I20260807 19:44:14.777168 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5408
Aug 07 19:44:14 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:44:14.792950 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.777952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.0.198.194:46499)
0807 19:44:14.778151 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.778158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.778179 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:44:14.778296 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.778300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.778391 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.778475 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:14.779976 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.780719 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.782099 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.782498 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.782595 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.786987 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:14.787004 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:14.787019 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:14.787060 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:14.790110 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.790798 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.790806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.790808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.790938 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.791265 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.791281 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.791283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.791808 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:14.792050 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.792325 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.792507 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:44:14.793628 5536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:14.887908 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5408
I20260807 19:44:14.910501 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5547
Aug 07 19:44:14 dist-test-slave-btw8 krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:44:14.929101 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:14.911495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.0.198.195:37547)
0807 19:44:14.911753 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:14.911762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:14.911875 (+ 113us) server_negotiation.cc:407] Connection header received
0807 19:44:14.912127 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:14.912133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:14.912240 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:14.912380 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:14.913941 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.914724 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.916417 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:14.916900 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:14.917069 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:14.922356 (+ 5287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:14.922377 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:14.922395 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:14.922441 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:14.925584 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.926474 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.926483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.926486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.926644 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:14.927131 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:14.927136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:14.927138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:14.927755 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:14.928121 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:14.928474 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 19:44:14.928728 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:44:14.929860 5672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:15.023856 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5547
I20260807 19:44:15.044168 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5164
I20260807 19:44:15.045598 5237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:15.161048 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5164
2026-08-07T19:44:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4772 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T19:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:15Z Disabled control of system clock
I20260807 19:44:15.249008 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:43191
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:43191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:44:15.610756 5737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:15.611086 5737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:15.611174 5737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:15.618934 5737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:15.619074 5737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:15.619113 5737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:15.619164 5737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:15.619210 5737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:15.629051 5737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:43191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:15.630417 5737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:15.632359 5737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:15.646378 5746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:15.646450 5742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:15.646517 5743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:15.647723 5737 server_base.cc:1034] running on GCE node
I20260807 19:44:15.648772 5737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:15.650489 5737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:15.651785 5737 hybrid_clock.cc:648] HybridClock initialized: now 1786131855651729 us; error 65 us; skew 500 ppm
I20260807 19:44:15.652356 5737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:15.655493 5737 webserver.cc:460] Webserver started at http://127.0.198.254:33579/ using document root <none> and password file <none>
I20260807 19:44:15.656484 5737 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:15.656600 5737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:15.656908 5737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:15.660276 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "f3a07062d6584420a3f33a8a629e3155"
format_stamp: "Formatted at 2026-08-07 19:44:15 on dist-test-slave-btw8"
server_key: "66b932dee465a0893d144dfb980f295b"
server_key_iv: "72a6337db8f08301524c455d6dbb678a"
server_key_version: "encryptionkey@0"
I20260807 19:44:15.661185 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "f3a07062d6584420a3f33a8a629e3155"
format_stamp: "Formatted at 2026-08-07 19:44:15 on dist-test-slave-btw8"
server_key: "66b932dee465a0893d144dfb980f295b"
server_key_iv: "72a6337db8f08301524c455d6dbb678a"
server_key_version: "encryptionkey@0"
I20260807 19:44:15.667447 5737 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:44:15.672055 5752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:15.673836 5737 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:44:15.674093 5737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "f3a07062d6584420a3f33a8a629e3155"
format_stamp: "Formatted at 2026-08-07 19:44:15 on dist-test-slave-btw8"
server_key: "66b932dee465a0893d144dfb980f295b"
server_key_iv: "72a6337db8f08301524c455d6dbb678a"
server_key_version: "encryptionkey@0"
I20260807 19:44:15.674407 5737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:15.681427 5737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:15.682415 5737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:15.682782 5737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:15.705590 5737 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:43191
I20260807 19:44:15.705651 5804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:43191 every 8 connection(s)
I20260807 19:44:15.707036 5737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:15.708424 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5737
I20260807 19:44:15.708631 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:15.709476 795 external_mini_cluster.cc:1428] Setting key 4c9318f4ce4f8aa3173e67d1b2250371
I20260807 19:44:15.712179 5805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:15.720495 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:15.711793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.0.198.254:43191)
0807 19:44:15.712918 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:15.712933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:15.712987 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:44:15.715416 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:15.715451 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:15.716274 (+ 823us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:15.716830 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:15.718104 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:15.718114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:15.718132 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:15.718163 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:15.718279 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:15.718305 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:15.718553 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:15.718713 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 19:44:15.719061 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":403,"threads_started":1}
I20260807 19:44:15.727250 5805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: Bootstrap starting.
I20260807 19:44:15.730827 5805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:15.731971 5805 log.cc:824] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:15.735353 5805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: No bootstrap required, opened a new log
I20260807 19:44:15.743495 5805 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } }
I20260807 19:44:15.743917 5805 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:15.743996 5805 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a07062d6584420a3f33a8a629e3155, State: Initialized, Role: FOLLOWER
I20260807 19:44:15.744611 5805 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } }
I20260807 19:44:15.744895 5805 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:15.745002 5805 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:15.745134 5805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:15.747473 5805 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } }
I20260807 19:44:15.747980 5805 leader_election.cc:302] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a07062d6584420a3f33a8a629e3155; no voters:
I20260807 19:44:15.748762 5805 leader_election.cc:288] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:15.748971 5810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:15.750072 5810 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [term 1 LEADER]: Becoming Leader. State: Replica: f3a07062d6584420a3f33a8a629e3155, State: Running, Role: LEADER
I20260807 19:44:15.750727 5810 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } }
I20260807 19:44:15.751338 5805 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:15.754089 5811 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3a07062d6584420a3f33a8a629e3155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } } }
I20260807 19:44:15.754093 5812 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a07062d6584420a3f33a8a629e3155. Latest consensus state: current_term: 1 leader_uuid: "f3a07062d6584420a3f33a8a629e3155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a07062d6584420a3f33a8a629e3155" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 43191 } } }
I20260807 19:44:15.754961 5812 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:15.756388 5811 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:15.764993 5819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:15.771139 5819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:15.782290 5819 catalog_manager.cc:1347] Generated new cluster ID: 734e5a791c5442df89d07b63051700e3
I20260807 19:44:15.782435 5819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:15.833557 5819 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:15.834159 5819 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:15.845014 5819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: Generated new TSK 0
I20260807 19:44:15.846167 5819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:44:15.859973 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--builtin_ntp_servers=127.0.198.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:44:16.232645 5829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:16.232970 5829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:16.233120 5829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:16.240983 5829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:16.241103 5829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:16.241290 5829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:16.251478 5829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:16.253077 5829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:16.254990 5829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:16.270427 5834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:16.270354 5838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:16.270685 5835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:16.272565 5829 server_base.cc:1034] running on GCE node
I20260807 19:44:16.273320 5829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:16.274336 5829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:16.275547 5829 hybrid_clock.cc:648] HybridClock initialized: now 1786131856275505 us; error 42 us; skew 500 ppm
I20260807 19:44:16.275873 5829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:16.278455 5829 webserver.cc:460] Webserver started at http://127.0.198.193:43715/ using document root <none> and password file <none>
I20260807 19:44:16.279177 5829 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:16.279281 5829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:16.279740 5829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:16.283216 5829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "4c3c2f3f96a74870810cd47fa77631ea"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "67d624a4d1fe12e11e57e119fea26694"
server_key_iv: "88e682491bab0d32570acbcd7578aff1"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.284147 5829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "4c3c2f3f96a74870810cd47fa77631ea"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "67d624a4d1fe12e11e57e119fea26694"
server_key_iv: "88e682491bab0d32570acbcd7578aff1"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.289955 5829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:44:16.293744 5844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.295259 5829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:44:16.295483 5829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "4c3c2f3f96a74870810cd47fa77631ea"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "67d624a4d1fe12e11e57e119fea26694"
server_key_iv: "88e682491bab0d32570acbcd7578aff1"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.295740 5829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:16.301646 5829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:16.302516 5829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:16.302809 5829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:16.303979 5829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:16.305473 5829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:16.305595 5829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.305737 5829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:16.305833 5829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.345603 5829 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:32947
I20260807 19:44:16.345644 5959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:32947 every 8 connection(s)
I20260807 19:44:16.347195 5829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:16.350597 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5829
I20260807 19:44:16.350823 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:16.351804 795 external_mini_cluster.cc:1428] Setting key 4dfc0e8efbd438cb347dcb33d4884cbe
I20260807 19:44:16.355068 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--builtin_ntp_servers=127.0.198.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:44:16.357683 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:16.350076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:38833 (local address 127.0.198.254:43191)
0807 19:44:16.350651 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:16.350656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:16.352373 (+ 1717us) server_negotiation.cc:407] Connection header received
0807 19:44:16.354704 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:16.354710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:16.354785 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:16.354934 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:16.356670 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:16.356677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:16.356682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:16.356707 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:16.356771 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:16.356787 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:16.356950 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:16.357183 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:44:16.357268 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":315,"threads_started":1}
I20260807 19:44:16.358832 5963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:16.350806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43191 (local address 127.0.198.193:38833)
0807 19:44:16.352188 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:16.352231 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:16.354391 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:16.355271 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:16.355289 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:16.355477 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:16.356310 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:16.356312 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:16.356357 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:16.356461 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:16.356905 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:16.356919 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:16.356930 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:44:16.357305 (+ 375us) client_negotiation.cc:240] Negotiation successful
0807 19:44:16.357694 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":364,"threads_started":1}
I20260807 19:44:16.360877 5960 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43191
I20260807 19:44:16.361274 5960 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:16.361990 5960 heartbeater.cc:511] Master 127.0.198.254:43191 requested a full tablet report, sending...
I20260807 19:44:16.364832 5769 ts_manager.cc:194] Registered new tserver with Master: 4c3c2f3f96a74870810cd47fa77631ea (127.0.198.193:32947)
W20260807 19:44:16.713377 5964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:16.713660 5964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:16.713819 5964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:16.721355 5964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:16.721469 5964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:16.721647 5964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:16.731666 5964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:16.733098 5964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:16.735041 5964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:16.750108 5969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:16.750223 5970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:16.750252 5973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:16.751140 5964 server_base.cc:1034] running on GCE node
I20260807 19:44:16.751966 5964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:16.753156 5964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:16.754441 5964 hybrid_clock.cc:648] HybridClock initialized: now 1786131856754349 us; error 94 us; skew 500 ppm
I20260807 19:44:16.754853 5964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:16.757792 5964 webserver.cc:460] Webserver started at http://127.0.198.194:40961/ using document root <none> and password file <none>
I20260807 19:44:16.758646 5964 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:16.758846 5964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:16.759258 5964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:16.762640 5964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "0601bb50c4284a35bd2bc37f020be3f6"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "18b6268fae73cda30c0f971030890aed"
server_key_iv: "99570030e234fabff1230c462fcbe1b4"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.763718 5964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "0601bb50c4284a35bd2bc37f020be3f6"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "18b6268fae73cda30c0f971030890aed"
server_key_iv: "99570030e234fabff1230c462fcbe1b4"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.769942 5964 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:16.773880 5979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.775666 5964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:44:16.775879 5964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "0601bb50c4284a35bd2bc37f020be3f6"
format_stamp: "Formatted at 2026-08-07 19:44:16 on dist-test-slave-btw8"
server_key: "18b6268fae73cda30c0f971030890aed"
server_key_iv: "99570030e234fabff1230c462fcbe1b4"
server_key_version: "encryptionkey@0"
I20260807 19:44:16.776165 5964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:16.782557 5964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:16.783413 5964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:16.783677 5964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:16.784713 5964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:16.786223 5964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:16.786311 5964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.786386 5964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:16.786443 5964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:16.824556 5964 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:38527
I20260807 19:44:16.824630 6096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:38527 every 8 connection(s)
I20260807 19:44:16.825987 5964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:16.835839 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 5964
I20260807 19:44:16.836084 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:16.836390 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:16.828893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:39085 (local address 127.0.198.254:43191)
0807 19:44:16.829134 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:16.829141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:16.830722 (+ 1581us) server_negotiation.cc:407] Connection header received
0807 19:44:16.833402 (+ 2680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:16.833407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:16.833482 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:16.833582 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:16.835424 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:16.835437 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:16.835440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:16.835461 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:16.835498 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:16.835512 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:16.835652 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:16.835966 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 19:44:16.836057 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:44:16.836691 795 external_mini_cluster.cc:1428] Setting key 329c0ca58459e7892625bd3a1aa320c7
I20260807 19:44:16.837738 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:16.829443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43191 (local address 127.0.198.194:39085)
0807 19:44:16.830465 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:16.830512 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:16.833070 (+ 2558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:16.833939 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:16.833949 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:16.834086 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:16.834961 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:16.834964 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:16.835006 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:16.835116 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:16.835670 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:16.835683 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:16.835693 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:44:16.836041 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 19:44:16.836354 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":333,"threads_started":1}
I20260807 19:44:16.839475 6097 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43191
I20260807 19:44:16.839861 6097 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:16.840657 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--builtin_ntp_servers=127.0.198.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:44:16.840734 6097 heartbeater.cc:511] Master 127.0.198.254:43191 requested a full tablet report, sending...
I20260807 19:44:16.842612 5769 ts_manager.cc:194] Registered new tserver with Master: 0601bb50c4284a35bd2bc37f020be3f6 (127.0.198.194:38527)
W20260807 19:44:17.204452 6100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:17.204770 6100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:17.204936 6100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:17.213201 6100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:17.213323 6100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:17.213515 6100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:17.223034 6100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:17.224496 6100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:17.226500 6100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:17.241261 6106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:17.241233 6110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:17.241343 6107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:17.242105 6100 server_base.cc:1034] running on GCE node
I20260807 19:44:17.242758 6100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:17.243920 6100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:17.245218 6100 hybrid_clock.cc:648] HybridClock initialized: now 1786131857245158 us; error 49 us; skew 500 ppm
I20260807 19:44:17.245602 6100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:17.248307 6100 webserver.cc:460] Webserver started at http://127.0.198.195:38937/ using document root <none> and password file <none>
I20260807 19:44:17.249078 6100 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:17.249222 6100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:17.249565 6100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:17.252941 6100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "24c8ac803263467899a8a43f7ed739aa"
format_stamp: "Formatted at 2026-08-07 19:44:17 on dist-test-slave-btw8"
server_key: "9771137d874d0f6cf76c1cc60cce9a01"
server_key_iv: "e906522a2182f2c5ada52a5d1938366c"
server_key_version: "encryptionkey@0"
I20260807 19:44:17.253865 6100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "24c8ac803263467899a8a43f7ed739aa"
format_stamp: "Formatted at 2026-08-07 19:44:17 on dist-test-slave-btw8"
server_key: "9771137d874d0f6cf76c1cc60cce9a01"
server_key_iv: "e906522a2182f2c5ada52a5d1938366c"
server_key_version: "encryptionkey@0"
I20260807 19:44:17.259976 6100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:44:17.263938 6117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:17.265523 6100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:44:17.265748 6100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "24c8ac803263467899a8a43f7ed739aa"
format_stamp: "Formatted at 2026-08-07 19:44:17 on dist-test-slave-btw8"
server_key: "9771137d874d0f6cf76c1cc60cce9a01"
server_key_iv: "e906522a2182f2c5ada52a5d1938366c"
server_key_version: "encryptionkey@0"
I20260807 19:44:17.266019 6100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:17.272524 6100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:17.273450 6100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:17.273756 6100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:17.274825 6100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:17.276489 6100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:17.276618 6100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:17.276765 6100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:17.276854 6100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:17.313638 6100 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:43913
I20260807 19:44:17.313700 6237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:43913 every 8 connection(s)
I20260807 19:44:17.315022 6100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:17.322068 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6100
I20260807 19:44:17.322300 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:17.322981 795 external_mini_cluster.cc:1428] Setting key bd5b3957ad672546dd4636ec26e4b02b
I20260807 19:44:17.323977 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.317816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:37797 (local address 127.0.198.254:43191)
0807 19:44:17.318019 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.318024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.319475 (+ 1451us) server_negotiation.cc:407] Connection header received
0807 19:44:17.321229 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.321234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.321297 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.321415 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.323048 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.323052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.323056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.323074 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.323142 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.323162 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.323289 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.323599 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.323662 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:44:17.325676 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.318346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:43191 (local address 127.0.198.195:37797)
0807 19:44:17.319269 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.319310 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.320920 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.321631 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.321639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.321768 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.322628 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.322632 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.322694 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.322831 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.323301 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.323314 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.323322 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.323715 (+ 393us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.323990 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":308,"threads_started":1}
I20260807 19:44:17.327530 6238 heartbeater.cc:347] Connected to a master server at 127.0.198.254:43191
I20260807 19:44:17.327940 6238 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:17.328661 6238 heartbeater.cc:511] Master 127.0.198.254:43191 requested a full tablet report, sending...
I20260807 19:44:17.330593 5768 ts_manager.cc:194] Registered new tserver with Master: 24c8ac803263467899a8a43f7ed739aa (127.0.198.195:43913)
I20260807 19:44:17.341006 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:44:17.351481 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.347757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.0.198.254:43191)
0807 19:44:17.348034 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.348045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.348811 (+ 766us) server_negotiation.cc:407] Connection header received
0807 19:44:17.349057 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.349063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.349135 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.349262 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.350557 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.350563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.350566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.350586 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.350628 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.350642 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.350771 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.350895 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.350956 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:44:17.357596 5768 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:17.366544 5768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:17.368685 5960 heartbeater.cc:503] Master 127.0.198.254:43191 was elected leader, sending a full tablet report...
I20260807 19:44:17.403501 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.381984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.0.198.195:43913)
0807 19:44:17.399061 (+ 17077us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.399073 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.399131 (+ 58us) server_negotiation.cc:407] Connection header received
0807 19:44:17.399227 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.399236 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.399565 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.399753 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.402442 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.402450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.402462 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.402478 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.402529 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.402545 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.402758 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.402934 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.403047 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16877,"thread_start_us":17386,"threads_started":1}
I20260807 19:44:17.403673 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.382539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:43913 (local address 127.0.0.1:52616)
0807 19:44:17.397234 (+ 14695us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.397274 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.397485 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.401232 (+ 3747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.401239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.401281 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.402098 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.402103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.402138 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.402234 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.402663 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.402676 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.402684 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.402959 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.403055 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14401,"thread_start_us":16003,"threads_started":1}
I20260807 19:44:17.404944 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.383411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.0.198.193:32947)
0807 19:44:17.398473 (+ 15062us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.398486 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.399171 (+ 685us) server_negotiation.cc:407] Connection header received
0807 19:44:17.399250 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.399257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.399571 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.399747 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.402496 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.402507 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.402521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.402543 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.402606 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.402629 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.402877 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.404451 (+ 1574us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.404553 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14850,"thread_start_us":14651,"threads_started":1}
I20260807 19:44:17.405671 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.382002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.0.198.194:38527)
0807 19:44:17.402601 (+ 20599us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.402610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.402642 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:44:17.402719 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.402728 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.402984 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.403909 (+ 925us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.404808 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.404818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.404835 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.404858 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.404911 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.404934 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.405087 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.405242 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.405336 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20401,"thread_start_us":19744,"threads_started":1}
I20260807 19:44:17.406039 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.383031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43086)
0807 19:44:17.398877 (+ 15846us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.398901 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.399026 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.400760 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.400770 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.400885 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.402098 (+ 1213us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.402103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.402138 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.402234 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.403209 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.403213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.403218 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.403564 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.405282 (+ 1718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15695,"thread_start_us":14616,"threads_started":1}
I20260807 19:44:17.406278 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.381806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38527 (local address 127.0.0.1:47042)
0807 19:44:17.382581 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.382645 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.382794 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.403201 (+ 20407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.403206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.403233 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.404599 (+ 1366us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.404602 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.404619 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.404664 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.405060 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.405067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.405073 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.405233 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.405288 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":344,"threads_started":1}
I20260807 19:44:17.408144 6168 tablet_service.cc:1467] Processing CreateTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 (DEFAULT_TABLE table=test-table [id=25fc88b7088e46168867ddabfaecd724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:17.409682 6168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94facf9ba7234c77bbdb1932034e5cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:17.409780 6029 tablet_service.cc:1467] Processing CreateTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 (DEFAULT_TABLE table=test-table [id=25fc88b7088e46168867ddabfaecd724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:17.409780 5895 tablet_service.cc:1467] Processing CreateTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 (DEFAULT_TABLE table=test-table [id=25fc88b7088e46168867ddabfaecd724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:17.411167 6029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94facf9ba7234c77bbdb1932034e5cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:17.411173 5895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94facf9ba7234c77bbdb1932034e5cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:17.422406 6257 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Bootstrap starting.
I20260807 19:44:17.426347 6257 tablet_bootstrap.cc:654] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:17.427060 6258 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Bootstrap starting.
I20260807 19:44:17.428005 6257 log.cc:824] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:17.429683 6259 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Bootstrap starting.
I20260807 19:44:17.431303 6258 tablet_bootstrap.cc:654] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:17.431988 6257 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: No bootstrap required, opened a new log
I20260807 19:44:17.432350 6257 ts_tablet_manager.cc:1397] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:44:17.432832 6258 log.cc:824] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:17.435216 6259 tablet_bootstrap.cc:654] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:17.436374 6259 log.cc:824] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:17.439864 6259 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: No bootstrap required, opened a new log
I20260807 19:44:17.440214 6259 ts_tablet_manager.cc:1397] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:44:17.449138 6259 raft_consensus.cc:348] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.449607 6259 raft_consensus.cc:374] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:17.449759 6259 raft_consensus.cc:729] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3c2f3f96a74870810cd47fa77631ea, State: Initialized, Role: FOLLOWER
I20260807 19:44:17.449741 6257 raft_consensus.cc:348] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.450284 6257 raft_consensus.cc:374] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:17.450404 6257 raft_consensus.cc:729] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c8ac803263467899a8a43f7ed739aa, State: Initialized, Role: FOLLOWER
I20260807 19:44:17.450585 6259 consensus_queue.cc:260] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.451262 6258 tablet_bootstrap.cc:492] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: No bootstrap required, opened a new log
I20260807 19:44:17.451092 6257 consensus_queue.cc:260] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.451793 6258 ts_tablet_manager.cc:1397] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.007s
I20260807 19:44:17.456669 6259 ts_tablet_manager.cc:1428] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Time spent starting tablet: real 0.016s user 0.008s sys 0.005s
I20260807 19:44:17.459262 6238 heartbeater.cc:503] Master 127.0.198.254:43191 was elected leader, sending a full tablet report...
I20260807 19:44:17.460525 6257 ts_tablet_manager.cc:1428] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Time spent starting tablet: real 0.028s user 0.021s sys 0.004s
I20260807 19:44:17.463929 6258 raft_consensus.cc:348] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.464510 6258 raft_consensus.cc:374] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:17.464679 6258 raft_consensus.cc:729] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0601bb50c4284a35bd2bc37f020be3f6, State: Initialized, Role: FOLLOWER
I20260807 19:44:17.465431 6258 consensus_queue.cc:260] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.466728 6097 heartbeater.cc:503] Master 127.0.198.254:43191 was elected leader, sending a full tablet report...
I20260807 19:44:17.467651 6258 ts_tablet_manager.cc:1428] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
W20260807 19:44:17.566978 6239 tablet.cc:2369] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:17.577924 6264 raft_consensus.cc:482] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:44:17.578452 6098 tablet.cc:2369] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:17.578339 6264 raft_consensus.cc:504] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.580686 6264 leader_election.cc:288] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0601bb50c4284a35bd2bc37f020be3f6 (127.0.198.194:38527), 4c3c2f3f96a74870810cd47fa77631ea (127.0.198.193:32947)
I20260807 19:44:17.583835 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.581173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:38527 (local address 127.0.198.195:47801)
0807 19:44:17.581423 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.581448 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.581601 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.582154 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.582158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.582178 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.582868 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.582871 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.582886 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.582937 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.583334 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.583339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.583345 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.583497 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.583553 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:44:17.583856 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.581304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:47801 (local address 127.0.198.194:38527)
0807 19:44:17.581531 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.581537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.581564 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:44:17.581779 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.581784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.581856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.581965 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.583091 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.583097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.583102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.583125 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.583171 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.583187 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.583324 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.583508 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.583562 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:44:17.584594 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.581576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.198.195:41383)
0807 19:44:17.582249 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.582268 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:17.582416 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:17.582755 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:17.582759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:17.582781 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:17.583500 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:44:17.583504 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:17.583521 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:44:17.583562 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:17.584053 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:17.584058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:17.584063 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:44:17.584197 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 19:44:17.584240 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":347,"threads_started":1}
I20260807 19:44:17.584798 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.582028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:41383 (local address 127.0.198.193:32947)
0807 19:44:17.582258 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.582265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.582364 (+ 99us) server_negotiation.cc:407] Connection header received
0807 19:44:17.582544 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.582550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.582610 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.582712 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.583718 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.583724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.583729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.583748 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.583788 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.583803 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.583942 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.584499 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.584551 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:44:17.585044 6050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94facf9ba7234c77bbdb1932034e5cc7" candidate_uuid: "24c8ac803263467899a8a43f7ed739aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0601bb50c4284a35bd2bc37f020be3f6" is_pre_election: true
I20260807 19:44:17.585611 6050 raft_consensus.cc:2454] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c8ac803263467899a8a43f7ed739aa in term 0.
I20260807 19:44:17.586390 6121 leader_election.cc:302] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0601bb50c4284a35bd2bc37f020be3f6, 24c8ac803263467899a8a43f7ed739aa; no voters:
I20260807 19:44:17.586835 6264 raft_consensus.cc:2781] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:17.586992 6264 raft_consensus.cc:482] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:17.587159 6264 raft_consensus.cc:3037] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:17.587044 5915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94facf9ba7234c77bbdb1932034e5cc7" candidate_uuid: "24c8ac803263467899a8a43f7ed739aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3c2f3f96a74870810cd47fa77631ea" is_pre_election: true
I20260807 19:44:17.587677 5915 raft_consensus.cc:2454] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c8ac803263467899a8a43f7ed739aa in term 0.
I20260807 19:44:17.590144 6264 raft_consensus.cc:504] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
I20260807 19:44:17.590942 6264 leader_election.cc:288] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [CANDIDATE]: Term 1 election: Requested vote from peers 0601bb50c4284a35bd2bc37f020be3f6 (127.0.198.194:38527), 4c3c2f3f96a74870810cd47fa77631ea (127.0.198.193:32947)
I20260807 19:44:17.591614 6050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94facf9ba7234c77bbdb1932034e5cc7" candidate_uuid: "24c8ac803263467899a8a43f7ed739aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0601bb50c4284a35bd2bc37f020be3f6"
I20260807 19:44:17.591696 5915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94facf9ba7234c77bbdb1932034e5cc7" candidate_uuid: "24c8ac803263467899a8a43f7ed739aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c3c2f3f96a74870810cd47fa77631ea"
I20260807 19:44:17.591966 6050 raft_consensus.cc:3037] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:17.591964 5915 raft_consensus.cc:3037] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:17.595542 6050 raft_consensus.cc:2454] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c8ac803263467899a8a43f7ed739aa in term 1.
I20260807 19:44:17.595541 5915 raft_consensus.cc:2454] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c8ac803263467899a8a43f7ed739aa in term 1.
I20260807 19:44:17.596138 6121 leader_election.cc:302] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0601bb50c4284a35bd2bc37f020be3f6, 24c8ac803263467899a8a43f7ed739aa; no voters:
I20260807 19:44:17.596555 6264 raft_consensus.cc:2781] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:17.597328 6264 raft_consensus.cc:686] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 1 LEADER]: Becoming Leader. State: Replica: 24c8ac803263467899a8a43f7ed739aa, State: Running, Role: LEADER
I20260807 19:44:17.598031 6264 consensus_queue.cc:237] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } }
W20260807 19:44:17.599948 5961 tablet.cc:2369] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:17.603650 5768 catalog_manager.cc:5495] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24c8ac803263467899a8a43f7ed739aa (127.0.198.195). New cstate: current_term: 1 leader_uuid: "24c8ac803263467899a8a43f7ed739aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c8ac803263467899a8a43f7ed739aa" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 43913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:17.660465 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.652520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.0.198.195:43913)
0807 19:44:17.652828 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.652836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.652867 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:44:17.652976 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.652980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.653068 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.653179 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.659867 (+ 6688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.659873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.659877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.659901 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.659951 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.659968 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.660136 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.660181 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.660246 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260807 19:44:17.670307 6050 raft_consensus.cc:1261] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 1 FOLLOWER]: Refusing update from remote peer 24c8ac803263467899a8a43f7ed739aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:17.670307 5915 raft_consensus.cc:1261] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 1 FOLLOWER]: Refusing update from remote peer 24c8ac803263467899a8a43f7ed739aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:17.671718 6264 consensus_queue.cc:1035] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0601bb50c4284a35bd2bc37f020be3f6" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:17.672130 6267 consensus_queue.cc:1035] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c3c2f3f96a74870810cd47fa77631ea" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:17.685258 6270 mvcc.cc:205] Tried to move back new op lower bound from 7315996089004122112 to 7315996088726396928. Current Snapshot: MvccSnapshot[applied={T|T < 7315996089004122112}]
I20260807 19:44:17.686908 6272 mvcc.cc:205] Tried to move back new op lower bound from 7315996089004122112 to 7315996088726396928. Current Snapshot: MvccSnapshot[applied={T|T < 7315996089004122112}]
I20260807 19:44:17.694028 6271 mvcc.cc:205] Tried to move back new op lower bound from 7315996089004122112 to 7315996088726396928. Current Snapshot: MvccSnapshot[applied={T|T < 7315996089004122112}]
I20260807 19:44:17.705681 5768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:17.706039 5768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:17.710856 5768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a07062d6584420a3f33a8a629e3155: Sending DeleteTablet for 3 replicas of tablet 94facf9ba7234c77bbdb1932034e5cc7
I20260807 19:44:17.712415 6029 tablet_service.cc:1514] Processing DeleteTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:17 UTC) from {username='slave'} at 127.0.0.1:47042
I20260807 19:44:17.712847 5895 tablet_service.cc:1514] Processing DeleteTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:17 UTC) from {username='slave'} at 127.0.0.1:43086
I20260807 19:44:17.713336 6168 tablet_service.cc:1514] Processing DeleteTablet for tablet 94facf9ba7234c77bbdb1932034e5cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:17 UTC) from {username='slave'} at 127.0.0.1:52616
I20260807 19:44:17.716514 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5829
I20260807 19:44:17.724115 6282 tablet_replica.cc:330] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: stopping tablet replica
I20260807 19:44:17.724716 6284 tablet_replica.cc:330] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: stopping tablet replica
I20260807 19:44:17.724714 6282 raft_consensus.cc:2229] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:17.725165 6284 raft_consensus.cc:2229] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:17.725310 6282 raft_consensus.cc:2258] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:17.725940 6284 raft_consensus.cc:2258] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:17.728333 6283 tablet_replica.cc:330] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: stopping tablet replica
I20260807 19:44:17.728423 6282 ts_tablet_manager.cc:1905] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:17.728793 6283 raft_consensus.cc:2229] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:17.729472 6283 raft_consensus.cc:2258] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:17.731721 6283 ts_tablet_manager.cc:1905] T 94facf9ba7234c77bbdb1932034e5cc7 P 4c3c2f3f96a74870810cd47fa77631ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:17.735503 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.718427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.0.198.193:32947)
0807 19:44:17.718646 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.718654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.732330 (+ 13676us) server_negotiation.cc:407] Connection header received
0807 19:44:17.732667 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.732674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.732772 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.732864 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.734507 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.734514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.734520 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.734547 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.734593 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.734614 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.734745 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.735030 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.735105 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:44:17.736963 5955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:17.739457 6284 ts_tablet_manager.cc:1905] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:17.746964 6282 ts_tablet_manager.cc:1918] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:17.747265 6282 log.cc:1196] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/94facf9ba7234c77bbdb1932034e5cc7
I20260807 19:44:17.749089 6284 ts_tablet_manager.cc:1918] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:17.749459 6284 log.cc:1196] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/94facf9ba7234c77bbdb1932034e5cc7
I20260807 19:44:17.750170 6284 ts_tablet_manager.cc:1939] T 94facf9ba7234c77bbdb1932034e5cc7 P 24c8ac803263467899a8a43f7ed739aa: Deleting consensus metadata
I20260807 19:44:17.752555 5753 catalog_manager.cc:4841] TS 24c8ac803263467899a8a43f7ed739aa (127.0.198.195:43913): tablet 94facf9ba7234c77bbdb1932034e5cc7 (table test-table [id=25fc88b7088e46168867ddabfaecd724]) successfully deleted
I20260807 19:44:17.753809 6282 ts_tablet_manager.cc:1939] T 94facf9ba7234c77bbdb1932034e5cc7 P 0601bb50c4284a35bd2bc37f020be3f6: Deleting consensus metadata
I20260807 19:44:17.756165 5756 catalog_manager.cc:4841] TS 0601bb50c4284a35bd2bc37f020be3f6 (127.0.198.194:38527): tablet 94facf9ba7234c77bbdb1932034e5cc7 (table test-table [id=25fc88b7088e46168867ddabfaecd724]) successfully deleted
I20260807 19:44:17.932835 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5829
W20260807 19:44:17.955657 5756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:32947 (error 108)
I20260807 19:44:17.956512 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5964
I20260807 19:44:17.958207 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.957214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43100)
0807 19:44:17.957924 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:17.958043 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":416,"threads_started":1}
W20260807 19:44:17.958469 5756 catalog_manager.cc:4584] TS 4c3c2f3f96a74870810cd47fa77631ea (127.0.198.193:32947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94facf9ba7234c77bbdb1932034e5cc7: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
I20260807 19:44:17.960585 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:17.957443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47048 (local address 127.0.198.194:38527)
0807 19:44:17.958144 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:17.958151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:17.958172 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:44:17.958271 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:17.958278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:17.958401 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:17.958513 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:17.959839 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:17.959846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:17.959853 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:17.959880 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:17.959927 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:17.959946 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:17.960101 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:17.960175 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 19:44:17.960244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537}
I20260807 19:44:17.961545 6092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:18.012621 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.012009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43106)
0807 19:44:18.012291 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.012424 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:44:18.014001 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.013533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43108)
0807 19:44:18.013713 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.013832 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:44:18.057596 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5964
I20260807 19:44:18.074862 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.074258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43114)
0807 19:44:18.074534 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.074663 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 19:44:18.076433 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.075827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43130)
0807 19:44:18.076088 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.076207 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:44:18.084280 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6100
I20260807 19:44:18.092371 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.085285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.0.198.195:43913)
0807 19:44:18.085472 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:18.085479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:18.085501 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:44:18.085722 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:18.085728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:18.085797 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:18.085935 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:18.091151 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:18.091158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:18.091163 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:18.091192 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:18.091259 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:18.091305 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:18.091814 (+ 509us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:18.091898 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 19:44:18.091977 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:44:18.093279 6233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:18.157889 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.157292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43140)
0807 19:44:18.157552 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.157671 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 19:44:18.159320 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.158730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43150)
0807 19:44:18.158986 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.159083 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 19:44:18.221261 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6100
I20260807 19:44:18.246047 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 5737
I20260807 19:44:18.247437 5800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:18.360682 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.360131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32947 (local address 127.0.0.1:43166)
0807 19:44:18.360354 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:18.360464 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:44:18.361047 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 5737
2026-08-07T19:44:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3200 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 19:44:18.411700 795 mini_oidc.cc:152] Starting JWKS server
I20260807 19:44:18.414848 795 webserver.cc:460] Webserver started at https://127.0.0.1:36543/ using document root <none> and password file <none>
I20260807 19:44:18.415298 795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:44:18.416200 795 webserver.cc:460] Webserver started at http://127.0.0.1:42167/ using document root <none> and password file <none>
2026-08-07T19:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:18Z Disabled control of system clock
I20260807 19:44:18.458556 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34701
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34701
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36543/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/testchainca.cert with env {}
W20260807 19:44:18.824488 6298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:18.824918 6298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:18.825116 6298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:18.833698 6298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:18.833885 6298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:18.833984 6298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:18.834107 6298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:18.834200 6298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:44:18.834295 6298 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:44:18.834369 6298 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36543/jwks.json
I20260807 19:44:18.845806 6298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36543/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:18.847419 6298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:18.848949 6298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:18.863276 6303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:18.863476 6304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:18.863523 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:18.877797 6298 server_base.cc:1034] running on GCE node
I20260807 19:44:18.878778 6298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:18.880472 6298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:18.881747 6298 hybrid_clock.cc:648] HybridClock initialized: now 1786131858881682 us; error 49 us; skew 500 ppm
I20260807 19:44:18.882198 6298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:18.884975 6298 webserver.cc:460] Webserver started at http://127.0.198.254:40451/ using document root <none> and password file <none>
I20260807 19:44:18.885766 6298 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:18.885871 6298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:18.886386 6298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:18.889876 6298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "658c395dd96c488cbaa07db1f9c74292"
format_stamp: "Formatted at 2026-08-07 19:44:18 on dist-test-slave-btw8"
server_key: "c653a2008d71217c098dde3db4c11560"
server_key_iv: "a7c4eddf4b091cfb7684093025798d38"
server_key_version: "encryptionkey@0"
I20260807 19:44:18.890813 6298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "658c395dd96c488cbaa07db1f9c74292"
format_stamp: "Formatted at 2026-08-07 19:44:18 on dist-test-slave-btw8"
server_key: "c653a2008d71217c098dde3db4c11560"
server_key_iv: "a7c4eddf4b091cfb7684093025798d38"
server_key_version: "encryptionkey@0"
I20260807 19:44:18.896600 6298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:44:18.900488 6313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:18.902041 6298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:44:18.902256 6298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "658c395dd96c488cbaa07db1f9c74292"
format_stamp: "Formatted at 2026-08-07 19:44:18 on dist-test-slave-btw8"
server_key: "c653a2008d71217c098dde3db4c11560"
server_key_iv: "a7c4eddf4b091cfb7684093025798d38"
server_key_version: "encryptionkey@0"
I20260807 19:44:18.902510 6298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:18.921684 6298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:18.922886 6298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:18.923309 6298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:18.944088 6298 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:34701
I20260807 19:44:18.944101 6366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:34701 every 8 connection(s)
I20260807 19:44:18.945638 6298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:18.948485 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6298
I20260807 19:44:18.948746 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:18.949597 795 external_mini_cluster.cc:1428] Setting key ec79882aa75b0b5623a7f4179eeb3f4a
I20260807 19:44:18.950449 6367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:18.966084 6367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292: Bootstrap starting.
I20260807 19:44:18.967005 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:18.951817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.0.198.254:34701)
0807 19:44:18.952919 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:18.952931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:18.952979 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:44:18.955862 (+ 2883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:18.955894 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:18.956558 (+ 664us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:18.956997 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:18.957823 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:18.959595 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:18.961774 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:18.962568 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:18.962748 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:18.964305 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:18.964313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:18.964330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:18.964984 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:18.965069 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:18.965094 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:18.965190 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:18.965461 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 19:44:18.965892 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":426,"threads_started":1}
I20260807 19:44:18.969642 6367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:18.970819 6367 log.cc:824] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:18.971544 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:44:18.974531 6367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292: No bootstrap required, opened a new log
I20260807 19:44:18.982317 6367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } }
I20260807 19:44:18.982810 6367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:18.982936 6367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658c395dd96c488cbaa07db1f9c74292, State: Initialized, Role: FOLLOWER
I20260807 19:44:18.983644 6367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } }
I20260807 19:44:18.983923 6367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:18.984079 6367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:18.984270 6367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:18.987133 6367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } }
I20260807 19:44:18.988122 6367 leader_election.cc:302] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 658c395dd96c488cbaa07db1f9c74292; no voters:
I20260807 19:44:18.988997 6367 leader_election.cc:288] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:18.989144 6373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:18.989938 6373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [term 1 LEADER]: Becoming Leader. State: Replica: 658c395dd96c488cbaa07db1f9c74292, State: Running, Role: LEADER
I20260807 19:44:18.990643 6373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } }
I20260807 19:44:18.991848 6367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:18.994217 6374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "658c395dd96c488cbaa07db1f9c74292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } } }
I20260807 19:44:18.994800 6374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:18.994701 6375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 658c395dd96c488cbaa07db1f9c74292. Latest consensus state: current_term: 1 leader_uuid: "658c395dd96c488cbaa07db1f9c74292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658c395dd96c488cbaa07db1f9c74292" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34701 } } }
I20260807 19:44:18.995507 6375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:19.002175 6381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:19.008423 6381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:19.019598 6381 catalog_manager.cc:1347] Generated new cluster ID: 1da34c76459844c6990fcab550c967e4
I20260807 19:44:19.019750 6381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:19.030097 6381 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:19.031695 6381 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:19.042868 6381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 658c395dd96c488cbaa07db1f9c74292: Generated new TSK 0
I20260807 19:44:19.044210 6381 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:44:19.097150 6399 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 19:44:19.097426 6399 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:44:19.098790 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:19.073995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55236 (local address 127.0.198.254:34701)
0807 19:44:19.074201 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:19.074207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:19.074666 (+ 459us) server_negotiation.cc:407] Connection header received
0807 19:44:19.075054 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:19.075061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:19.075165 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:19.075375 (+ 210us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:44:19.076044 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.077191 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.078801 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.079540 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.079692 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:19.080312 (+ 620us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:44:19.097979 (+ 17667us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:44:19.098428 (+ 449us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55236: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:44:19.098938 6371 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55236: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 19:44:19.100445 6398 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:19.073827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34701 (local address 127.0.0.1:55236)
0807 19:44:19.074539 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:19.074557 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:19.074711 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:19.075484 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:19.075489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:19.075501 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:44:19.075870 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:19.075879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.077371 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.077376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:19.078630 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:19.078643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.079720 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.079761 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:19.079897 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:19.079970 (+ 73us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:44:19.098377 (+ 18407us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:44:19.098674 (+ 297us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.198.254:34701: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":368,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.198.254:34701: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36543/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 19:44:19.102877 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6298
I20260807 19:44:19.104300 6362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:19.202539 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6298
2026-08-07T19:44:19Z chronyd exiting
I20260807 19:44:19.258203 795 test_util.cc:175] -----------------------------------------------
I20260807 19:44:19.258412 795 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131811070045-795-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (853 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 19:44:19.262988 795 mini_oidc.cc:152] Starting JWKS server
I20260807 19:44:19.265304 795 webserver.cc:460] Webserver started at https://127.0.0.1:37117/ using document root <none> and password file <none>
I20260807 19:44:19.265602 795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:44:19.266381 795 webserver.cc:460] Webserver started at http://127.0.0.1:43929/ using document root <none> and password file <none>
2026-08-07T19:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:19Z Disabled control of system clock
I20260807 19:44:19.309506 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:42267
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:42267
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37117/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/testchainca.cert with env {}
W20260807 19:44:19.662921 6406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:19.663273 6406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:19.663466 6406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:19.671489 6406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:19.671631 6406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:19.671672 6406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:19.671883 6406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:19.672077 6406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:44:19.672221 6406 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:44:19.672317 6406 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37117/jwks.json
I20260807 19:44:19.683604 6406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:42267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:42267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37117/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:19.685290 6406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:19.686918 6406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:19.701177 6411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:19.701725 6412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:19.701149 6414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:19.702824 6406 server_base.cc:1034] running on GCE node
I20260807 19:44:19.703814 6406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:19.705375 6406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:19.706602 6406 hybrid_clock.cc:648] HybridClock initialized: now 1786131859706543 us; error 57 us; skew 500 ppm
I20260807 19:44:19.707044 6406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:19.709784 6406 webserver.cc:460] Webserver started at http://127.0.198.254:42837/ using document root <none> and password file <none>
I20260807 19:44:19.710546 6406 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:19.710646 6406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:19.710987 6406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:19.714144 6406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "702b8681324b413ea85b30e02cbf3ae2"
format_stamp: "Formatted at 2026-08-07 19:44:19 on dist-test-slave-btw8"
server_key: "4a5db93a2fedddba0fe6486d73229ce8"
server_key_iv: "d2691fa73e8d440a63ad5c4f2ea76eed"
server_key_version: "encryptionkey@0"
I20260807 19:44:19.715052 6406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "702b8681324b413ea85b30e02cbf3ae2"
format_stamp: "Formatted at 2026-08-07 19:44:19 on dist-test-slave-btw8"
server_key: "4a5db93a2fedddba0fe6486d73229ce8"
server_key_iv: "d2691fa73e8d440a63ad5c4f2ea76eed"
server_key_version: "encryptionkey@0"
I20260807 19:44:19.721066 6406 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260807 19:44:19.725528 6420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:19.727481 6406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:19.727895 6406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "702b8681324b413ea85b30e02cbf3ae2"
format_stamp: "Formatted at 2026-08-07 19:44:19 on dist-test-slave-btw8"
server_key: "4a5db93a2fedddba0fe6486d73229ce8"
server_key_iv: "d2691fa73e8d440a63ad5c4f2ea76eed"
server_key_version: "encryptionkey@0"
I20260807 19:44:19.728211 6406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:19.754398 6406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:19.755671 6406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:19.756117 6406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:19.779034 6406 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:42267
I20260807 19:44:19.779050 6473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:42267 every 8 connection(s)
I20260807 19:44:19.780700 6406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:19.785140 6474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:19.788376 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6406
I20260807 19:44:19.788611 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:19.789338 795 external_mini_cluster.cc:1428] Setting key 6077931005c7f79025cc62475908b6c2
I20260807 19:44:19.798079 6474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2: Bootstrap starting.
I20260807 19:44:19.801496 6474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:19.802625 6474 log.cc:824] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:19.804533 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:19.791521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.0.198.254:42267)
0807 19:44:19.792683 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:19.792695 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:19.792744 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:44:19.794589 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:19.794612 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:19.795145 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:19.795696 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:19.796360 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.797697 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.799188 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.800032 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.800268 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:19.801558 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:19.801567 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:19.801580 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:19.802248 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:19.802357 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:19.802383 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:19.802538 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:19.802785 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 19:44:19.803108 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":390,"threads_started":1}
I20260807 19:44:19.806237 6474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2: No bootstrap required, opened a new log
I20260807 19:44:19.808945 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:44:19.814383 6474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } }
I20260807 19:44:19.814787 6474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:19.814895 6474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702b8681324b413ea85b30e02cbf3ae2, State: Initialized, Role: FOLLOWER
I20260807 19:44:19.815572 6474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } }
I20260807 19:44:19.815834 6474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:19.815989 6474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:19.816177 6474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:19.818480 6474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } }
I20260807 19:44:19.819155 6474 leader_election.cc:302] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 702b8681324b413ea85b30e02cbf3ae2; no voters:
I20260807 19:44:19.819805 6474 leader_election.cc:288] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:19.820019 6479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:19.820791 6479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [term 1 LEADER]: Becoming Leader. State: Replica: 702b8681324b413ea85b30e02cbf3ae2, State: Running, Role: LEADER
I20260807 19:44:19.821337 6479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } }
I20260807 19:44:19.822059 6474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:19.824571 6480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "702b8681324b413ea85b30e02cbf3ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } } }
I20260807 19:44:19.824544 6481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 702b8681324b413ea85b30e02cbf3ae2. Latest consensus state: current_term: 1 leader_uuid: "702b8681324b413ea85b30e02cbf3ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702b8681324b413ea85b30e02cbf3ae2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42267 } } }
I20260807 19:44:19.825356 6481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:19.826962 6480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:19.832902 6485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:19.837577 6485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:19.846840 6485 catalog_manager.cc:1347] Generated new cluster ID: e1f17fbb8f034f2ebb00388d1328119b
I20260807 19:44:19.846987 6485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:19.859728 6485 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:19.861191 6485 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:19.873163 6485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 702b8681324b413ea85b30e02cbf3ae2: Generated new TSK 0
I20260807 19:44:19.874342 6485 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:44:19.918277 6504 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 19:44:19.918447 6504 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:44:19.919972 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:19.894646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45384 (local address 127.0.198.254:42267)
0807 19:44:19.894869 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:19.894876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:19.895346 (+ 470us) server_negotiation.cc:407] Connection header received
0807 19:44:19.895688 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:19.895694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:19.895818 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:19.896017 (+ 199us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:44:19.896537 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.897759 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.899348 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.900125 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.900284 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:19.901157 (+ 873us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:44:19.919232 (+ 18075us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37117/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:44:19.919718 (+ 486us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45384: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37117/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:44:19.920164 6477 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45384: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37117/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 19:44:19.920425 6503 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:19.894487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:42267 (local address 127.0.0.1:45384)
0807 19:44:19.895229 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:19.895251 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:19.895461 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:19.896036 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:19.896039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:19.896050 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:44:19.896357 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:19.896366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.897949 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.897953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:19.899169 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:19.899184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:19.900489 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:19.900497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:19.900717 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:19.900774 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:44:19.919586 (+ 18812us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37117/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:44:19.919921 (+ 335us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.198.254:42267: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37117/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":408,"threads_started":1}
I20260807 19:44:19.921993 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6406
I20260807 19:44:19.923511 6468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:20.022711 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6406
2026-08-07T19:44:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (820 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 19:44:20.083230 795 mini_oidc.cc:152] Starting JWKS server
I20260807 19:44:20.085494 795 webserver.cc:460] Webserver started at https://127.0.0.1:44741/ using document root <none> and password file <none>
I20260807 19:44:20.085752 795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:44:20.086534 795 webserver.cc:460] Webserver started at http://127.0.0.1:43341/ using document root <none> and password file <none>
2026-08-07T19:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:20Z Disabled control of system clock
I20260807 19:44:20.127681 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40525
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:40525
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44741/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:44:20.484870 6511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:20.485250 6511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:20.485419 6511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:20.493211 6511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:20.493402 6511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:20.493498 6511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:20.493614 6511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:20.493693 6511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:44:20.493798 6511 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:44:20.493878 6511 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44741/jwks.json
I20260807 19:44:20.504251 6511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:40525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44741/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:20.505733 6511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:20.507725 6511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:20.521181 6517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:20.521368 6518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:20.521694 6521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:20.523108 6511 server_base.cc:1034] running on GCE node
I20260807 19:44:20.524075 6511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:20.525779 6511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:20.527073 6511 hybrid_clock.cc:648] HybridClock initialized: now 1786131860527006 us; error 41 us; skew 500 ppm
I20260807 19:44:20.527580 6511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:20.530555 6511 webserver.cc:460] Webserver started at http://127.0.198.254:34243/ using document root <none> and password file <none>
I20260807 19:44:20.531335 6511 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:20.531535 6511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:20.531874 6511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:20.535211 6511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "73210a0065604ce389a35d6b0401f4c8"
format_stamp: "Formatted at 2026-08-07 19:44:20 on dist-test-slave-btw8"
server_key: "5009d90b0bc9732c68a99999b783dde2"
server_key_iv: "df9db1a4e33aa731abd873892f9b8a73"
server_key_version: "encryptionkey@0"
I20260807 19:44:20.536302 6511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "73210a0065604ce389a35d6b0401f4c8"
format_stamp: "Formatted at 2026-08-07 19:44:20 on dist-test-slave-btw8"
server_key: "5009d90b0bc9732c68a99999b783dde2"
server_key_iv: "df9db1a4e33aa731abd873892f9b8a73"
server_key_version: "encryptionkey@0"
I20260807 19:44:20.543224 6511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:44:20.548203 6528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:20.549830 6511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260807 19:44:20.550089 6511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "73210a0065604ce389a35d6b0401f4c8"
format_stamp: "Formatted at 2026-08-07 19:44:20 on dist-test-slave-btw8"
server_key: "5009d90b0bc9732c68a99999b783dde2"
server_key_iv: "df9db1a4e33aa731abd873892f9b8a73"
server_key_version: "encryptionkey@0"
I20260807 19:44:20.550392 6511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:20.575263 6511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:20.576272 6511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:20.576637 6511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:20.597276 6511 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:40525
I20260807 19:44:20.597321 6582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:40525 every 8 connection(s)
I20260807 19:44:20.598783 6511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:20.602890 6583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:20.607877 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6511
I20260807 19:44:20.608139 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:20.609023 795 external_mini_cluster.cc:1428] Setting key 7a23f32121e359064283b3b39da9f7c8
I20260807 19:44:20.615981 6583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8: Bootstrap starting.
I20260807 19:44:20.619565 6583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:20.621188 6583 log.cc:824] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:20.625433 6583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8: No bootstrap required, opened a new log
I20260807 19:44:20.626107 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:20.611439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42608 (local address 127.0.198.254:40525)
0807 19:44:20.612717 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:20.612729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:20.612786 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:44:20.614789 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:20.614816 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:20.615447 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:20.615927 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:20.616641 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.618143 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.619905 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.621105 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.621351 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:20.622866 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:20.622879 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:20.622903 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:20.623722 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:20.623839 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:20.623870 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:20.624009 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:20.624326 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:44:20.624733 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":399,"threads_started":1}
I20260807 19:44:20.630683 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:44:20.634634 6583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } }
I20260807 19:44:20.635046 6583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:20.635178 6583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73210a0065604ce389a35d6b0401f4c8, State: Initialized, Role: FOLLOWER
I20260807 19:44:20.635888 6583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } }
I20260807 19:44:20.636145 6583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:20.636346 6583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:20.636546 6583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:20.639230 6583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } }
I20260807 19:44:20.639887 6583 leader_election.cc:302] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73210a0065604ce389a35d6b0401f4c8; no voters:
I20260807 19:44:20.640522 6583 leader_election.cc:288] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:20.640697 6588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:20.641494 6588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [term 1 LEADER]: Becoming Leader. State: Replica: 73210a0065604ce389a35d6b0401f4c8, State: Running, Role: LEADER
I20260807 19:44:20.642170 6588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } }
I20260807 19:44:20.642807 6583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:20.645344 6590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73210a0065604ce389a35d6b0401f4c8. Latest consensus state: current_term: 1 leader_uuid: "73210a0065604ce389a35d6b0401f4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } } }
I20260807 19:44:20.645956 6590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:20.645975 6589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73210a0065604ce389a35d6b0401f4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73210a0065604ce389a35d6b0401f4c8" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40525 } } }
I20260807 19:44:20.646587 6589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:20.648332 6596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:20.654814 6596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:20.665244 6596 catalog_manager.cc:1347] Generated new cluster ID: 5c4019c33d314a9eb1da0e5eac8a1a8e
I20260807 19:44:20.665475 6596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:20.685563 6596 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:20.687557 6596 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:20.696059 6596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73210a0065604ce389a35d6b0401f4c8: Generated new TSK 0
I20260807 19:44:20.697230 6596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:44:20.730691 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:20.720864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.0.198.254:40525)
0807 19:44:20.721092 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:20.721099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:20.721693 (+ 594us) server_negotiation.cc:407] Connection header received
0807 19:44:20.721968 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:20.721975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:20.722092 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:20.722289 (+ 197us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:44:20.722834 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.724070 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.725751 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.726517 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.726676 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:20.727399 (+ 723us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:44:20.729969 (+ 2570us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:44:20.730345 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42622: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:44:20.730839 6586 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42622: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 19:44:20.732496 6614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:20.720830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40525 (local address 127.0.0.1:42622)
0807 19:44:20.721567 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:20.721586 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:20.721737 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:20.722325 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:20.722329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:20.722341 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:44:20.722626 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:20.722634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.724295 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.724300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:20.725533 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:20.725549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:20.726813 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:20.726819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:20.726965 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:20.727016 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:44:20.730233 (+ 3217us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:44:20.732057 (+ 1824us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.198.254:40525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":1446,"thread_start_us":406,"threads_started":1}
I20260807 19:44:20.733955 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6511
I20260807 19:44:20.735559 6578 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 19:44:20.745970 6615 webserver.cc:533] Webserver: sslize failed:
E20260807 19:44:20.746140 6615 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:44:20.834503 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6511
2026-08-07T19:44:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (811 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T19:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:20Z Disabled control of system clock
I20260807 19:44:20.936703 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:45131
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:45131
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/testchainca.cert with env {}
W20260807 19:44:21.295048 6622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:21.295428 6622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:21.295558 6622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:21.303071 6622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:21.303208 6622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:21.303248 6622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:21.303299 6622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:21.303344 6622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:44:21.303501 6622 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:44:21.303571 6622 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 19:44:21.314317 6622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:45131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:45131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:21.315846 6622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:21.317273 6622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:21.330773 6629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:21.330826 6630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:21.330866 6632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:21.336549 6622 server_base.cc:1034] running on GCE node
I20260807 19:44:21.337652 6622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:21.339507 6622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:21.340803 6622 hybrid_clock.cc:648] HybridClock initialized: now 1786131861340738 us; error 57 us; skew 500 ppm
I20260807 19:44:21.341236 6622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:44:21.344383 6622 webserver.cc:460] Webserver started at http://127.0.198.254:35007/ using document root <none> and password file <none>
I20260807 19:44:21.345232 6622 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:21.345392 6622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:21.345783 6622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:21.349233 6622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3"
format_stamp: "Formatted at 2026-08-07 19:44:21 on dist-test-slave-btw8"
server_key: "72d7182c195818f1b9af43e06692d333"
server_key_iv: "25da3af31d75cff8aaa9408ad12a4320"
server_key_version: "encryptionkey@0"
I20260807 19:44:21.350347 6622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3"
format_stamp: "Formatted at 2026-08-07 19:44:21 on dist-test-slave-btw8"
server_key: "72d7182c195818f1b9af43e06692d333"
server_key_iv: "25da3af31d75cff8aaa9408ad12a4320"
server_key_version: "encryptionkey@0"
I20260807 19:44:21.356603 6622 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:44:21.360762 6638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:21.362605 6622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260807 19:44:21.362885 6622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3"
format_stamp: "Formatted at 2026-08-07 19:44:21 on dist-test-slave-btw8"
server_key: "72d7182c195818f1b9af43e06692d333"
server_key_iv: "25da3af31d75cff8aaa9408ad12a4320"
server_key_version: "encryptionkey@0"
I20260807 19:44:21.363168 6622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:21.398057 6622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:21.399001 6622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:21.399322 6622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:21.419916 6622 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:45131
I20260807 19:44:21.419961 6692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:45131 every 8 connection(s)
I20260807 19:44:21.421412 6622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:21.425940 6693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:21.426759 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6622
I20260807 19:44:21.426992 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:21.428035 795 external_mini_cluster.cc:1428] Setting key 58fd3206337232db938569ca4cb8f919
I20260807 19:44:21.443344 6693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3: Bootstrap starting.
I20260807 19:44:21.445740 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:21.430863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.0.198.254:45131)
0807 19:44:21.432974 (+ 2111us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:21.432986 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:21.433035 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:44:21.434749 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:21.434777 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:21.435343 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:21.435804 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:21.436462 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.437818 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.439905 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.440986 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.441184 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:21.442813 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:21.442823 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:21.442841 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:44:21.443609 (+ 768us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:21.443724 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:44:21.443752 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:21.443883 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:21.444183 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 19:44:21.444615 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1593,"thread_start_us":1278,"threads_started":1}
I20260807 19:44:21.447268 6693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:21.448529 6693 log.cc:824] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:21.450565 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:44:21.452697 6693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3: No bootstrap required, opened a new log
I20260807 19:44:21.461860 6693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } }
I20260807 19:44:21.462360 6693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:21.462448 6693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f7eb3d66b34e1e9935ec8c37ad42b3, State: Initialized, Role: FOLLOWER
I20260807 19:44:21.463012 6693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } }
I20260807 19:44:21.463231 6693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:21.463340 6693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:21.463577 6693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:21.465919 6693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } }
I20260807 19:44:21.466451 6693 leader_election.cc:302] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39f7eb3d66b34e1e9935ec8c37ad42b3; no voters:
I20260807 19:44:21.467101 6693 leader_election.cc:288] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:21.467288 6698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:21.468093 6698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [term 1 LEADER]: Becoming Leader. State: Replica: 39f7eb3d66b34e1e9935ec8c37ad42b3, State: Running, Role: LEADER
I20260807 19:44:21.468783 6698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } }
I20260807 19:44:21.469615 6693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:21.471909 6699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } } }
I20260807 19:44:21.472301 6699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:21.473413 6704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:21.473235 6700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39f7eb3d66b34e1e9935ec8c37ad42b3. Latest consensus state: current_term: 1 leader_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f7eb3d66b34e1e9935ec8c37ad42b3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45131 } } }
I20260807 19:44:21.473611 6700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:21.480214 6704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:21.492274 6704 catalog_manager.cc:1347] Generated new cluster ID: 8ae40c128fa04f3f98d20cc88175400a
I20260807 19:44:21.492435 6704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:21.504390 6704 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:21.505934 6704 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:21.520403 6704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39f7eb3d66b34e1e9935ec8c37ad42b3: Generated new TSK 0
I20260807 19:44:21.521414 6704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:44:21.607756 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:21.534716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.0.198.254:45131)
0807 19:44:21.534988 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:21.534995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:21.535493 (+ 498us) server_negotiation.cc:407] Connection header received
0807 19:44:21.535829 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:21.535836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:21.535932 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:21.536128 (+ 196us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:44:21.536782 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.537560 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.539981 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.540510 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.540702 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:21.541398 (+ 696us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:44:21.607033 (+ 65635us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:44:21.607488 (+ 455us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37158: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:44:21.607926 6696 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37158: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 19:44:21.608564 6723 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:21.534605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45131 (local address 127.0.0.1:37158)
0807 19:44:21.535306 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:21.535331 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:21.535552 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:21.536160 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:21.536167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:21.536182 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:44:21.536553 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:21.536570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.537740 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.537746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:21.539798 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:21.539817 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:21.540801 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:21.540809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:21.541014 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:21.541077 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:44:21.607404 (+ 66327us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:44:21.607780 (+ 376us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.198.254:45131: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":345,"threads_started":1}
I20260807 19:44:21.609846 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6622
I20260807 19:44:21.611817 6686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:21.716456 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6622
2026-08-07T19:44:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (863 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (468 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (446 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:22 dist-test-slave-btw8 krb5kdc[6739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:22 dist-test-slave-btw8 krb5kdc[6739](info): set up 2 sockets
Aug 07 19:44:22 dist-test-slave-btw8 krb5kdc[6739](info): commencing operation
krb5kdc: starting...
W20260807 19:44:23.870260 795 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:24 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:24.172091 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36097
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:36097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:24.535738 6753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:24.536095 6753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:24.536193 6753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:24.544044 6753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:24.544183 6753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:24.544231 6753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:24.544281 6753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:24.544328 6753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:24.555154 6753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:36097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:24.556694 6753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:24.558043 6753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:24.571815 6758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:24.571786 6759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:24.572371 6762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:24.572965 6753 server_base.cc:1034] running on GCE node
I20260807 19:44:24.573894 6753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:24.575666 6753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:24.576993 6753 hybrid_clock.cc:648] HybridClock initialized: now 1786131864576927 us; error 58 us; skew 500 ppm
I20260807 19:44:24.577430 6753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:24 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:24.585044 6753 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:24.586886 6753 webserver.cc:460] Webserver started at http://127.0.198.254:43901/ using document root <none> and password file <none>
I20260807 19:44:24.587740 6753 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:24.587867 6753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:24.588162 6753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:24.591609 6753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "b2ae4922a3bf40c2b5637d975b68b484"
format_stamp: "Formatted at 2026-08-07 19:44:24 on dist-test-slave-btw8"
server_key: "8d338fd7743bd841b430c240c65facec"
server_key_iv: "aa55faa2e41fcb5bfe227a1684c5213a"
server_key_version: "encryptionkey@0"
I20260807 19:44:24.592518 6753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "b2ae4922a3bf40c2b5637d975b68b484"
format_stamp: "Formatted at 2026-08-07 19:44:24 on dist-test-slave-btw8"
server_key: "8d338fd7743bd841b430c240c65facec"
server_key_iv: "aa55faa2e41fcb5bfe227a1684c5213a"
server_key_version: "encryptionkey@0"
I20260807 19:44:24.598399 6753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:44:24.602694 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:24.604375 6753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:24.604550 6753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "b2ae4922a3bf40c2b5637d975b68b484"
format_stamp: "Formatted at 2026-08-07 19:44:24 on dist-test-slave-btw8"
server_key: "8d338fd7743bd841b430c240c65facec"
server_key_iv: "aa55faa2e41fcb5bfe227a1684c5213a"
server_key_version: "encryptionkey@0"
I20260807 19:44:24.604749 6753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:24.646049 6753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:24.651051 6753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:24.651422 6753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:24.673098 6753 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:36097
I20260807 19:44:24.673164 6824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:36097 every 8 connection(s)
I20260807 19:44:24.675127 6753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:24.679682 6825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:24.683893 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6753
I20260807 19:44:24.684145 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:24.684875 795 external_mini_cluster.cc:1428] Setting key a719a5fd5e11f26b9e1ae86aec7586c6
I20260807 19:44:24.693521 6825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484: Bootstrap starting.
I20260807 19:44:24.697635 6825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:24.699115 6825 log.cc:824] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:44:24 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:24.703148 6825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484: No bootstrap required, opened a new log
I20260807 19:44:24.714771 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:24.687204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.0.198.254:36097)
0807 19:44:24.688156 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:24.688174 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:24.688226 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:44:24.691440 (+ 3214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:24.691475 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:24.692094 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:24.692609 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:24.694108 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:24.696251 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:24.697600 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:24.698758 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:24.698968 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:24.704452 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:24.704466 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:24.704486 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:24.704543 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:24.709059 (+ 4516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:24.710148 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:24.710161 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:24.710172 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:24.710444 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:24.710966 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:24.710970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:24.710973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:24.711862 (+ 889us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:24.712191 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:24.712652 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:44:24.713187 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":434,"threads_started":1}
I20260807 19:44:24.714680 6825 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } }
I20260807 19:44:24.715209 6825 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:24.715334 6825 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2ae4922a3bf40c2b5637d975b68b484, State: Initialized, Role: FOLLOWER
I20260807 19:44:24.716133 6825 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } }
I20260807 19:44:24.716463 6825 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:24.716655 6825 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:24.716869 6825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:24.720108 6825 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } }
I20260807 19:44:24.720821 6825 leader_election.cc:302] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2ae4922a3bf40c2b5637d975b68b484; no voters:
I20260807 19:44:24.721907 6825 leader_election.cc:288] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:24.722049 6830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:24.722754 6830 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [term 1 LEADER]: Becoming Leader. State: Replica: b2ae4922a3bf40c2b5637d975b68b484, State: Running, Role: LEADER
I20260807 19:44:24.723562 6830 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } }
I20260807 19:44:24.725258 6825 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:24.726801 6831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ae4922a3bf40c2b5637d975b68b484. Latest consensus state: current_term: 1 leader_uuid: "b2ae4922a3bf40c2b5637d975b68b484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } } }
I20260807 19:44:24.727531 6831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:24.727012 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2ae4922a3bf40c2b5637d975b68b484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ae4922a3bf40c2b5637d975b68b484" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36097 } } }
I20260807 19:44:24.728032 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:24.728811 6836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:24.743678 6836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:24.754858 6836 catalog_manager.cc:1347] Generated new cluster ID: a012fe127f4c4368b19c64797e770645
I20260807 19:44:24.755038 6836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:24.778710 6836 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:24.780293 6836 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:24.790833 6836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2ae4922a3bf40c2b5637d975b68b484: Generated new TSK 0
I20260807 19:44:24.792136 6836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:44:24.861119 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:25.231592 6853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:25.231909 6853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:25.232086 6853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:25.240025 6853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:25.240223 6853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:25.240440 6853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:25.251151 6853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:25.252684 6853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:25.254115 6853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:25.271255 6858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:25.271405 6859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:25.271440 6861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:25.272604 6853 server_base.cc:1034] running on GCE node
I20260807 19:44:25.273414 6853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:25.274587 6853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:25.275861 6853 hybrid_clock.cc:648] HybridClock initialized: now 1786131865275815 us; error 40 us; skew 500 ppm
I20260807 19:44:25.276271 6853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:25 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:25.279884 6853 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:44:25.281550 6853 webserver.cc:460] Webserver started at http://127.0.198.193:33545/ using document root <none> and password file <none>
I20260807 19:44:25.282331 6853 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:25.282482 6853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:25.282815 6853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:25.285956 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "d812ce2e78a24dbd92c0f56fb87a1a5f"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "a921ed8a32b1efde59e33981dd28511d"
server_key_iv: "0d97446862b2f61a3f0a42861d68ad85"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.286798 6853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "d812ce2e78a24dbd92c0f56fb87a1a5f"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "a921ed8a32b1efde59e33981dd28511d"
server_key_iv: "0d97446862b2f61a3f0a42861d68ad85"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.292484 6853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:44:25.296429 6868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.298008 6853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:44:25.298240 6853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "d812ce2e78a24dbd92c0f56fb87a1a5f"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "a921ed8a32b1efde59e33981dd28511d"
server_key_iv: "0d97446862b2f61a3f0a42861d68ad85"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.298485 6853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:25.332964 6853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:25.337865 6853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:25.338204 6853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:25.339326 6853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:25.341063 6853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:25.341172 6853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.341346 6853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:25.341424 6853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.378122 6853 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:45793
I20260807 19:44:25.378187 6985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:45793 every 8 connection(s)
I20260807 19:44:25.379621 6853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:25.381982 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6853
I20260807 19:44:25.382220 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:25.383126 795 external_mini_cluster.cc:1428] Setting key 830bc7a0189bc5f473c913abf7027b37
Aug 07 19:44:25 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:25.402838 6988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:25.382811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:50827 (local address 127.0.198.254:36097)
0807 19:44:25.383347 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:25.383402 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:25.384711 (+ 1309us) server_negotiation.cc:407] Connection header received
0807 19:44:25.386819 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:25.386825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:25.386913 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:25.387042 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:25.388800 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.389556 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.391213 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.391767 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.391915 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:25.396102 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:25.396109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:25.396113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:25.396154 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:25.398895 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:25.400110 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:25.400114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:25.400117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:25.400255 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:25.400803 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:25.400806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:25.400808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:25.401077 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:25.401195 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:25.402318 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0807 19:44:25.402518 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":272,"threads_started":1}
I20260807 19:44:25.404675 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:25.383512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36097 (local address 127.0.198.193:50827)
0807 19:44:25.384520 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:25.384583 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:25.386531 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:25.387313 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:25.387321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:25.387997 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:25.388607 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:25.388626 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.389740 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.389745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:25.391049 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:25.391064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.392039 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.392048 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:25.392316 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:25.393111 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:25.393141 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:25.395725 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:25.399098 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:25.399114 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:25.399140 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:25.399957 (+ 817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:25.400377 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:25.400385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:25.400391 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:25.400676 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:25.401214 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:25.401225 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:25.401850 (+ 625us) client_negotiation.cc:769] Sending connection context
0807 19:44:25.402185 (+ 335us) client_negotiation.cc:240] Negotiation successful
0807 19:44:25.402864 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":419,"threads_started":1}
I20260807 19:44:25.406575 6986 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36097
I20260807 19:44:25.406947 6986 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:25.407871 6986 heartbeater.cc:511] Master 127.0.198.254:36097 requested a full tablet report, sending...
I20260807 19:44:25.411031 6788 ts_manager.cc:194] Registered new tserver with Master: d812ce2e78a24dbd92c0f56fb87a1a5f (127.0.198.193:45793)
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:44:25.413554 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:50827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:44:25.451804 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:25.820407 6993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:25.820721 6993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:25.820822 6993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:25.828600 6993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:25.828754 6993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:25.828898 6993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:25.839299 6993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:25.840799 6993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:25.842248 6993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:25.856698 6998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:25.856812 6999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:25.856738 7001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:25.858071 6993 server_base.cc:1034] running on GCE node
I20260807 19:44:25.858748 6993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:25.859946 6993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:25.861232 6993 hybrid_clock.cc:648] HybridClock initialized: now 1786131865861180 us; error 42 us; skew 500 ppm
I20260807 19:44:25.861613 6993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:25 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:25.865303 6993 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:44:25.867033 6993 webserver.cc:460] Webserver started at http://127.0.198.194:39565/ using document root <none> and password file <none>
I20260807 19:44:25.867866 6993 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:25.867986 6993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:25.868278 6993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:25.871603 6993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "2524e383181a464192a0921c3bb302f2"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "21b96b4e6519e1c2869b07d965765bf4"
server_key_iv: "e590369393b5e56c1b71c8509cf91356"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.872431 6993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "2524e383181a464192a0921c3bb302f2"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "21b96b4e6519e1c2869b07d965765bf4"
server_key_iv: "e590369393b5e56c1b71c8509cf91356"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.878309 6993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 19:44:25.882319 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.884146 6993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:25.884373 6993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "2524e383181a464192a0921c3bb302f2"
format_stamp: "Formatted at 2026-08-07 19:44:25 on dist-test-slave-btw8"
server_key: "21b96b4e6519e1c2869b07d965765bf4"
server_key_iv: "e590369393b5e56c1b71c8509cf91356"
server_key_version: "encryptionkey@0"
I20260807 19:44:25.884657 6993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:25.923136 6993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:25.928459 6993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:25.928799 6993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:25.929888 6993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:25.931572 6993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:25.931675 6993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.931842 6993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:25.931919 6993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:25.969795 6993 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:46105
I20260807 19:44:25.969859 7125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:46105 every 8 connection(s)
I20260807 19:44:25.971297 6993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:25.972378 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 6993
I20260807 19:44:25.972568 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:25.973160 795 external_mini_cluster.cc:1428] Setting key 0b9341644f33cbe8acb12df34f5c71de
Aug 07 19:44:25 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:25.996153 7128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:25.974554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:39939 (local address 127.0.198.254:36097)
0807 19:44:25.975142 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:25.975151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:25.976531 (+ 1380us) server_negotiation.cc:407] Connection header received
0807 19:44:25.978780 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:25.978789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:25.978893 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:25.979024 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:25.980807 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.981977 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.983863 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.984621 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.984791 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:25.988993 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:25.989003 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:25.989012 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:25.989066 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:25.992417 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:25.993389 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:25.993394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:25.993397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:25.993540 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:25.994080 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:25.994085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:25.994087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:25.994409 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:25.994588 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:25.995479 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 19:44:25.995752 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":274,"threads_started":1}
I20260807 19:44:25.997617 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:25.975242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36097 (local address 127.0.198.194:39939)
0807 19:44:25.976340 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:25.976392 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:25.978414 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:25.979255 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:25.979265 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:25.979960 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:25.980588 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:25.980609 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.982176 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.982181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:25.983675 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:25.983692 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:25.984910 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:25.984915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:25.985142 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:25.985946 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:25.985971 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:25.988581 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:25.992633 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:25.992644 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:25.992666 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:25.993206 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:25.993674 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:25.993679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:25.993683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:25.993928 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:25.994563 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:25.994572 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:25.995031 (+ 459us) client_negotiation.cc:769] Sending connection context
0807 19:44:25.995344 (+ 313us) client_negotiation.cc:240] Negotiation successful
0807 19:44:25.996028 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":504,"threads_started":1}
I20260807 19:44:25.999464 7126 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36097
I20260807 19:44:25.999807 7126 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:26.000623 7126 heartbeater.cc:511] Master 127.0.198.254:36097 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
I20260807 19:44:26.002468 6788 ts_manager.cc:194] Registered new tserver with Master: 2524e383181a464192a0921c3bb302f2 (127.0.198.194:46105)
I20260807 19:44:26.003612 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:39939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:44:26.039862 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:26.414099 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:26.414446 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:26.414549 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:44:26.417271 6986 heartbeater.cc:503] Master 127.0.198.254:36097 was elected leader, sending a full tablet report...
W20260807 19:44:26.422837 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:26.422976 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:26.423177 7133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:26.433948 7133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:26.435456 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:26.436872 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:26.452554 7142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:26.452605 7139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:26.452521 7138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:26.454440 7133 server_base.cc:1034] running on GCE node
I20260807 19:44:26.455294 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:26.456480 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:26.457790 7133 hybrid_clock.cc:648] HybridClock initialized: now 1786131866457713 us; error 36 us; skew 500 ppm
I20260807 19:44:26.458174 7133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.461655 7133 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:44:26.463426 7133 webserver.cc:460] Webserver started at http://127.0.198.195:33953/ using document root <none> and password file <none>
I20260807 19:44:26.464169 7133 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:26.464322 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:26.464684 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:26.468314 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "2ecfa98ce45e4cc195bb859b7d83bc21"
format_stamp: "Formatted at 2026-08-07 19:44:26 on dist-test-slave-btw8"
server_key: "92626bda132a967fe48bbcaa892b3b51"
server_key_iv: "6e46204393c2e62fdafe2586b0610106"
server_key_version: "encryptionkey@0"
I20260807 19:44:26.469163 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "2ecfa98ce45e4cc195bb859b7d83bc21"
format_stamp: "Formatted at 2026-08-07 19:44:26 on dist-test-slave-btw8"
server_key: "92626bda132a967fe48bbcaa892b3b51"
server_key_iv: "6e46204393c2e62fdafe2586b0610106"
server_key_version: "encryptionkey@0"
I20260807 19:44:26.474975 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:44:26.479274 7149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:26.481127 7133 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 19:44:26.481387 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "2ecfa98ce45e4cc195bb859b7d83bc21"
format_stamp: "Formatted at 2026-08-07 19:44:26 on dist-test-slave-btw8"
server_key: "92626bda132a967fe48bbcaa892b3b51"
server_key_iv: "6e46204393c2e62fdafe2586b0610106"
server_key_version: "encryptionkey@0"
I20260807 19:44:26.481665 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:26.497414 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:26.502462 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:26.502880 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:26.504202 7133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:26.505820 7133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:26.505934 7133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:26.506091 7133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:26.506165 7133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:26.548157 7133 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36117
I20260807 19:44:26.548214 7265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36117 every 8 connection(s)
I20260807 19:44:26.549623 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:26.550920 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7133
I20260807 19:44:26.551155 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:26.551908 795 external_mini_cluster.cc:1428] Setting key b84841f03900bc55cea19680a301117b
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:26.573789 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.553042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:40947 (local address 127.0.198.254:36097)
0807 19:44:26.553600 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.553605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.554942 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 19:44:26.557330 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.557335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.557435 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.557568 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.559515 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.560340 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.562059 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.562461 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.562584 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.566854 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.566863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.566869 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.566916 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.570196 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.571110 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.571116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.571118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.571289 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.571911 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.571915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.571918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.572253 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.572392 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.573273 (+ 881us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.573478 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":291,"threads_started":1}
I20260807 19:44:26.574896 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.553626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36097 (local address 127.0.198.195:40947)
0807 19:44:26.554694 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:26.554785 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:26.556989 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:26.557878 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:26.557888 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:26.558552 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:26.559248 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:26.559268 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.560503 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.560507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:26.561895 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:26.561909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.562744 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.562751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:26.563007 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.563789 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:26.563825 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:26.566582 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:26.570341 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:26.570350 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:26.570367 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:26.570905 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:26.571520 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:26.571524 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:26.571527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:26.571788 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:26.572384 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:26.572391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:26.572845 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 19:44:26.573135 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 19:44:26.573584 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":502,"threads_started":1}
I20260807 19:44:26.576689 7266 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36097
I20260807 19:44:26.577119 7266 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:26.577975 7266 heartbeater.cc:511] Master 127.0.198.254:36097 requested a full tablet report, sending...
I20260807 19:44:26.579924 6788 ts_manager.cc:194] Registered new tserver with Master: 2ecfa98ce45e4cc195bb859b7d83bc21 (127.0.198.195:36117)
I20260807 19:44:26.581257 6788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:40947
I20260807 19:44:26.582763 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/test-admin.keytab.
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.621780 795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 19:44:26.628630 7274 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:26.625520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.0.198.193:45793)
0807 19:44:26.626563 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.626571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.626601 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:44:26.626668 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.626676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.626915 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.627057 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.628171 (+ 1114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34578: BlockingRecv error: recv got EOF from 127.0.0.1:34578 (error 108)
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":399,"threads_started":1}
W20260807 19:44:26.628924 795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.631181 795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.632903 795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:26.653092 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.634916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.0.198.193:45793)
0807 19:44:26.635152 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.635158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.635427 (+ 269us) server_negotiation.cc:407] Connection header received
0807 19:44:26.635680 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.635685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.635772 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.635875 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.637423 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.638495 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.640267 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.641028 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.641155 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.646193 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.646211 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.646226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.646297 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.649552 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.650492 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.650501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.650503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.650641 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.651113 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.651116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.651118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.651637 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.651886 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.652459 (+ 573us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.652759 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 19:44:26.660071 7274 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:26.657842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.0.198.193:45793)
0807 19:44:26.658147 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.658155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.658533 (+ 378us) server_negotiation.cc:407] Connection header received
0807 19:44:26.658720 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.658724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.658795 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.658880 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.659816 (+ 936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34604: BlockingRecv error: recv got EOF from 127.0.0.1:34604 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 19:44:26.660331 795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.662088 795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.663551 795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:26.681056 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.665085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.0.198.193:45793)
0807 19:44:26.665340 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.665347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.665545 (+ 198us) server_negotiation.cc:407] Connection header received
0807 19:44:26.665710 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.665714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.665784 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.665876 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.667225 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.667976 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.669404 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.669836 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.669945 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.674550 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.674556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.674560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.674603 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.677814 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.678634 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.678642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.678646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.678812 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.679317 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.679323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.679327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.679798 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.680135 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.680458 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.680722 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 19:44:26.686203 7274 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:26.684146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.0.198.193:45793)
0807 19:44:26.684361 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.684368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.684634 (+ 266us) server_negotiation.cc:407] Connection header received
0807 19:44:26.684855 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.684860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.684934 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.685043 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.685922 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34614: BlockingRecv error: recv got EOF from 127.0.0.1:34614 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 19:44:26.686444 795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131811070045-795-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.688199 795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:26.689689 795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:26.705863 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.691096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.0.198.193:45793)
0807 19:44:26.691401 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.691408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.691615 (+ 207us) server_negotiation.cc:407] Connection header received
0807 19:44:26.691739 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.691743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.691803 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.691885 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.692928 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.693720 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.695146 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.695887 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.696038 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.699901 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.699906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.699909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.699947 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.702817 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.703720 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.703726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.703730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.703898 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.704316 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.704321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.704326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.704630 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.705012 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.705269 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.705513 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 19:44:26.707674 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6853
I20260807 19:44:26.720690 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.708498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.0.198.193:45793)
0807 19:44:26.708695 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.708700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.709083 (+ 383us) server_negotiation.cc:407] Connection header received
0807 19:44:26.709241 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.709244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.709338 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.709431 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.710480 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.711239 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.712577 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.712998 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.713099 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.715311 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.715316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.715319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.715421 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.718021 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.718778 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.718782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.718785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.718931 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.719283 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.719287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.719289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.719751 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.719954 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.720248 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.720428 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:44:26.721309 6981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:26.809057 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6853
I20260807 19:44:26.828836 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6993
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:44:26.847151 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.829872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.0.198.194:46105)
0807 19:44:26.830505 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.830512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.830538 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:44:26.830611 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.830618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.830833 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.830973 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.832211 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.833324 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.834864 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.835592 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.835712 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.840571 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.840586 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.840603 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.840645 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.843892 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.844745 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.844753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.844755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.844896 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.845281 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.845286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.845288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.845766 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.846047 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.846362 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.846702 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":341,"threads_started":1}
I20260807 19:44:26.848747 7120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:26.936254 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6993
I20260807 19:44:26.955699 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7133
Aug 07 19:44:26 dist-test-slave-btw8 krb5kdc[6739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:44:26.974509 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:26.956775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.0.198.195:36117)
0807 19:44:26.957457 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:26.957465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:26.957496 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:44:26.957561 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:26.957568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:26.957794 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:26.957929 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:26.959253 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.960392 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.961815 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:26.962473 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:26.962580 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:26.967450 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:26.967470 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:26.967487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:26.967534 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:26.970833 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.972112 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.972120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.972123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.972295 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:26.972783 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:26.972790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:26.972793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:26.973184 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:26.973456 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:26.973858 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 19:44:26.974173 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":331,"threads_started":1}
I20260807 19:44:26.976154 7261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:27.063527 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7133
I20260807 19:44:27.082746 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 6753
I20260807 19:44:27.084388 6820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:27.191967 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 6753
2026-08-07T19:44:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4565 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:27 dist-test-slave-btw8 krb5kdc[7296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:27 dist-test-slave-btw8 krb5kdc[7296](info): set up 2 sockets
Aug 07 19:44:27 dist-test-slave-btw8 krb5kdc[7296](info): commencing operation
krb5kdc: starting...
W20260807 19:44:28.389278 795 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:28 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab.
Entry for principal oryx/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab.
I20260807 19:44:28.688397 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39287
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab
--principal=oryx/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:29.041414 7310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:29.041839 7310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:29.042025 7310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:29.049713 7310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:29.049904 7310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:29.050006 7310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:29.050114 7310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:29.050202 7310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:29.060766 7310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab
--principal=oryx/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:29.062276 7310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:29.063684 7310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:29.076001 7318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:29.076022 7316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:29.076104 7315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:29.077091 7310 server_base.cc:1034] running on GCE node
I20260807 19:44:29.078009 7310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:29.079988 7310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:29.081234 7310 hybrid_clock.cc:648] HybridClock initialized: now 1786131869081186 us; error 39 us; skew 500 ppm
I20260807 19:44:29.081568 7310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:29 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:29.084913 7310 init.cc:377] Logged in from keytab as oryx/127.0.198.254@KRBTEST.COM (short username oryx)
I20260807 19:44:29.086695 7310 webserver.cc:460] Webserver started at http://127.0.198.254:41339/ using document root <none> and password file <none>
I20260807 19:44:29.087507 7310 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:29.087646 7310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:29.087976 7310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:29.091464 7310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "e0c2ec4c9adf47618b38e0316eb1472f"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "d3638412f66d5714af3cf0afce9d4cb2"
server_key_iv: "cce7f8d0bcc2214d6eb999d52a33dfde"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.092396 7310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "e0c2ec4c9adf47618b38e0316eb1472f"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "d3638412f66d5714af3cf0afce9d4cb2"
server_key_iv: "cce7f8d0bcc2214d6eb999d52a33dfde"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.098450 7310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 19:44:29.102568 7325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:29.104197 7310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:29.104401 7310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "e0c2ec4c9adf47618b38e0316eb1472f"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "d3638412f66d5714af3cf0afce9d4cb2"
server_key_iv: "cce7f8d0bcc2214d6eb999d52a33dfde"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.104606 7310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:29.133571 7310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:29.138998 7310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:29.139500 7310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:29.162421 7310 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:39287
I20260807 19:44:29.162452 7378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:39287 every 8 connection(s)
I20260807 19:44:29.164103 7310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:29.168180 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:29.168967 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7310
I20260807 19:44:29.169231 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:29.170089 795 external_mini_cluster.cc:1428] Setting key f949ae38dc477d3e8516da85e4b76698
I20260807 19:44:29.183802 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Bootstrap starting.
Aug 07 19:44:29 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:29.187327 7379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:29.188793 7379 log.cc:824] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:29.192853 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: No bootstrap required, opened a new log
I20260807 19:44:29.197865 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:29.172586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.0.198.254:39287)
0807 19:44:29.173514 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:29.173537 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:29.173593 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:44:29.175568 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:29.175602 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:29.176208 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:29.176692 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:29.178312 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.180477 (+ 2165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.181947 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.183018 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.183216 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:29.188111 (+ 4895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:29.188123 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:29.188143 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:29.188202 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:29.191999 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:29.193172 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:29.193185 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:29.193194 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:29.193480 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:29.193946 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:29.193953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:29.193957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:29.194819 (+ 862us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:29.195203 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:29.195575 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:44:29.196188 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":398,"threads_started":1}
I20260807 19:44:29.204049 7379 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } }
I20260807 19:44:29.204487 7379 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:29.204583 7379 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c2ec4c9adf47618b38e0316eb1472f, State: Initialized, Role: FOLLOWER
I20260807 19:44:29.205308 7379 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } }
I20260807 19:44:29.205621 7379 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:29.205826 7379 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:29.206063 7379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:29.208871 7379 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } }
I20260807 19:44:29.209546 7379 leader_election.cc:302] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c2ec4c9adf47618b38e0316eb1472f; no voters:
I20260807 19:44:29.213198 7379 leader_election.cc:288] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:29.213604 7385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:29.214449 7385 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 LEADER]: Becoming Leader. State: Replica: e0c2ec4c9adf47618b38e0316eb1472f, State: Running, Role: LEADER
I20260807 19:44:29.215008 7385 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } }
I20260807 19:44:29.216006 7379 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:29.218149 7387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c2ec4c9adf47618b38e0316eb1472f. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } }
I20260807 19:44:29.218874 7387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:29.219188 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } }
I20260807 19:44:29.221071 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:29.222555 7393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:29.228811 7393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:29.239831 7393 catalog_manager.cc:1347] Generated new cluster ID: a83fcd82a43c4b6a89bd1f0904f1191f
I20260807 19:44:29.240036 7393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:29.262686 7393 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:29.264833 7393 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:29.277891 7393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Generated new TSK 0
I20260807 19:44:29.279104 7393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
I20260807 19:44:29.353291 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:29.721020 7406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:29.721342 7406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:29.721561 7406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:29.729405 7406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:29.729555 7406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:29.729750 7406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:29.741817 7406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:29.743515 7406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:29.744973 7406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:29.759781 7412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:29.759881 7413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:29.760068 7415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:29.761221 7406 server_base.cc:1034] running on GCE node
I20260807 19:44:29.761891 7406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:29.762974 7406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:29.764204 7406 hybrid_clock.cc:648] HybridClock initialized: now 1786131869764149 us; error 44 us; skew 500 ppm
I20260807 19:44:29.764498 7406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:29 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:29.772111 7406 init.cc:377] Logged in from keytab as oryx/127.0.198.193@KRBTEST.COM (short username oryx)
I20260807 19:44:29.773821 7406 webserver.cc:460] Webserver started at http://127.0.198.193:37087/ using document root <none> and password file <none>
I20260807 19:44:29.774534 7406 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:29.774642 7406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:29.774919 7406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:29.778175 7406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "cadd11c386ff4d86872dd35e4e69414b"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "6801ba7139ab71ef50a3d07140bb6a0a"
server_key_iv: "d1f597f8188c78ae0ff0e858105bfbdd"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.779029 7406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "cadd11c386ff4d86872dd35e4e69414b"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "6801ba7139ab71ef50a3d07140bb6a0a"
server_key_iv: "d1f597f8188c78ae0ff0e858105bfbdd"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.785176 7406 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 19:44:29.788921 7422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:29.790416 7406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:44:29.790580 7406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "cadd11c386ff4d86872dd35e4e69414b"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "6801ba7139ab71ef50a3d07140bb6a0a"
server_key_iv: "d1f597f8188c78ae0ff0e858105bfbdd"
server_key_version: "encryptionkey@0"
I20260807 19:44:29.790781 7406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:29.815809 7406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:29.820835 7406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:29.821113 7406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:29.822211 7406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:29.823882 7406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:29.823962 7406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:29.824054 7406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:29.824103 7406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:29.860751 7406 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:32957
I20260807 19:44:29.860805 7539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:32957 every 8 connection(s)
I20260807 19:44:29.862164 7406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:29.865535 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7406
I20260807 19:44:29.865772 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:29.866715 795 external_mini_cluster.cc:1428] Setting key 422b905b13815bc57a89fa5b6a914020
Aug 07 19:44:29 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:29.886235 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:29.865037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:44547 (local address 127.0.198.254:39287)
0807 19:44:29.865623 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:29.865631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:29.866771 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 19:44:29.869234 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:29.869238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:29.869309 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:29.869404 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:29.871734 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.872474 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.874983 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.875495 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.875603 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:29.880050 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:29.880055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:29.880059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:29.880095 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:29.882787 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:29.883682 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:29.883687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:29.883689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:29.883818 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:29.884280 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:29.884288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:29.884294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:29.884629 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:29.884801 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:29.885589 (+ 788us) server_negotiation.cc:299] Negotiation successful
0807 19:44:29.885858 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":300,"threads_started":1}
I20260807 19:44:29.887290 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:29.865543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:44547)
0807 19:44:29.866538 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:29.866597 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:29.868874 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:29.869712 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:29.869724 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:29.870549 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:29.871549 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:29.871580 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.872656 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.872663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:29.874800 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:29.874821 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:29.875736 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:29.875746 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:29.876083 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:29.876867 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:29.876902 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:29.879794 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:29.882953 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:29.882963 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:29.882981 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:29.883544 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:29.883939 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:29.883944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:29.883948 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:29.884156 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:29.884773 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:29.884781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:29.885174 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 19:44:29.885470 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 19:44:29.886057 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":377,"threads_started":1}
I20260807 19:44:29.889288 7540 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:29.889722 7540 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:29.890544 7540 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:29.893011 7342 ts_manager.cc:194] Registered new tserver with Master: cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957)
WARNING: no policy specified for oryx/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:44:29.894608 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:44547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
I20260807 19:44:29.935894 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:30.286330 7546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:30.286641 7546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:30.286751 7546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:30.294628 7546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:30.294767 7546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:30.294937 7546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:30.305219 7546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:30.306589 7546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:30.307989 7546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:30.322994 7554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:30.323048 7552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:30.322956 7551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:30.329216 7546 server_base.cc:1034] running on GCE node
I20260807 19:44:30.330031 7546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:30.331176 7546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:30.332456 7546 hybrid_clock.cc:648] HybridClock initialized: now 1786131870332418 us; error 58 us; skew 500 ppm
I20260807 19:44:30.332834 7546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:30 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:30.336504 7546 init.cc:377] Logged in from keytab as oryx/127.0.198.194@KRBTEST.COM (short username oryx)
I20260807 19:44:30.338263 7546 webserver.cc:460] Webserver started at http://127.0.198.194:38361/ using document root <none> and password file <none>
I20260807 19:44:30.339133 7546 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:30.339293 7546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:30.339655 7546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:30.342926 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "dff9d30e63e343c4818e29779a7d06a3"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "ad712aaf94d5d3b5fb6b9174dfd24d29"
server_key_iv: "72651be26878f10257063f482de86278"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.343878 7546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "dff9d30e63e343c4818e29779a7d06a3"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "ad712aaf94d5d3b5fb6b9174dfd24d29"
server_key_iv: "72651be26878f10257063f482de86278"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.349845 7546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:30.354120 7562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.356043 7546 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:30.356249 7546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "dff9d30e63e343c4818e29779a7d06a3"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "ad712aaf94d5d3b5fb6b9174dfd24d29"
server_key_iv: "72651be26878f10257063f482de86278"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.356494 7546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:30.377077 7546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:30.381500 7546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:30.381844 7546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:30.382910 7546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:30.384636 7546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:30.384718 7546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.384877 7546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:30.384934 7546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.422081 7546 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:45393
I20260807 19:44:30.422129 7679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:45393 every 8 connection(s)
I20260807 19:44:30.423676 7546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:30.426833 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7546
I20260807 19:44:30.427030 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:30.427744 795 external_mini_cluster.cc:1428] Setting key 875b0085befff99fd141bb5ef5f86703
Aug 07 19:44:30 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:30.446872 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:30.426552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:47071 (local address 127.0.198.254:39287)
0807 19:44:30.427064 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:30.427069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:30.428354 (+ 1285us) server_negotiation.cc:407] Connection header received
0807 19:44:30.430452 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:30.430457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:30.430543 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:30.430675 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:30.432345 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:30.433089 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:30.434765 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:30.435269 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:30.435439 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:30.439768 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:30.439774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:30.439777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:30.439819 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:30.443018 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:30.444258 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:30.444263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:30.444265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:30.444416 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:30.445008 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:30.445011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:30.445013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:30.445291 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:30.445407 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:30.446403 (+ 996us) server_negotiation.cc:299] Negotiation successful
0807 19:44:30.446587 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":288,"threads_started":1}
I20260807 19:44:30.448556 7683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:30.427230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:47071)
0807 19:44:30.428151 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:30.428199 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:30.430165 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:30.430932 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:30.430940 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:30.431567 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:30.432153 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:30.432171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:30.433277 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:30.433281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:30.434590 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:30.434603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:30.435579 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:30.435587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:30.435871 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:30.436673 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:30.436701 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:30.439448 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:30.443230 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:30.443243 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:30.443267 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:30.444088 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:30.444562 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:30.444569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:30.444576 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:30.444883 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:30.445425 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:30.445436 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:30.445951 (+ 515us) client_negotiation.cc:769] Sending connection context
0807 19:44:30.446292 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 19:44:30.446995 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":324,"threads_started":1}
I20260807 19:44:30.450245 7680 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:30.450595 7680 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:30.451459 7680 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:30.453197 7342 ts_manager.cc:194] Registered new tserver with Master: dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393)
I20260807 19:44:30.454571 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:47071
WARNING: no policy specified for oryx/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
I20260807 19:44:30.496230 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:30.849722 7686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:30.850096 7686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:30.850282 7686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:30.858147 7686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:30.858340 7686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:30.858523 7686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:30.871441 7686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:30.872869 7686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:30.874392 7686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:30.889214 7692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:30.889135 7691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:30.889209 7694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:30.889990 7686 server_base.cc:1034] running on GCE node
I20260807 19:44:30.890939 7686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:30.892237 7686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:30.893535 7686 hybrid_clock.cc:648] HybridClock initialized: now 1786131870893490 us; error 57 us; skew 500 ppm
I20260807 19:44:30.893959 7686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:30 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:30.898089 7686 init.cc:377] Logged in from keytab as oryx/127.0.198.195@KRBTEST.COM (short username oryx)
I20260807 19:44:30.898285 7540 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
I20260807 19:44:30.900516 7686 webserver.cc:460] Webserver started at http://127.0.198.195:36277/ using document root <none> and password file <none>
I20260807 19:44:30.901350 7686 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:30.901473 7686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:30.901769 7686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:30.905215 7686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "e163a25c62af4e6e858c26742b591eca"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "641382c8d4129c6d990a58add9edff73"
server_key_iv: "b47a37e5378dbd88a4e3f08a3aea81be"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.906073 7686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "e163a25c62af4e6e858c26742b591eca"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "641382c8d4129c6d990a58add9edff73"
server_key_iv: "b47a37e5378dbd88a4e3f08a3aea81be"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.912540 7686 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:44:30.917182 7701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.919003 7686 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:44:30.919291 7686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "e163a25c62af4e6e858c26742b591eca"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "641382c8d4129c6d990a58add9edff73"
server_key_iv: "b47a37e5378dbd88a4e3f08a3aea81be"
server_key_version: "encryptionkey@0"
I20260807 19:44:30.919588 7686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:30.941890 7686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:30.947295 7686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:30.947691 7686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:30.948822 7686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:30.950536 7686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:30.950651 7686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.950822 7686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:30.950904 7686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:30.989883 7686 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:42655
I20260807 19:44:30.989920 7819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:42655 every 8 connection(s)
I20260807 19:44:30.991410 7686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:30.997167 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7686
I20260807 19:44:30.997386 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:30.997865 795 external_mini_cluster.cc:1428] Setting key 4e39a8e2fe38b647b3207287f3c7d559
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:31.016228 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:30.994528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:35425 (local address 127.0.198.254:39287)
0807 19:44:30.995191 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:30.995197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:30.996266 (+ 1069us) server_negotiation.cc:407] Connection header received
0807 19:44:30.998415 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:30.998421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:30.998554 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:30.998705 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.000706 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.001961 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.003678 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.004133 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.004268 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.008346 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.008352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.008355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.008394 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.011771 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.012949 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.012957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.012960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.013147 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.013967 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.013976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.013980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.014459 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.014651 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.015703 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.015902 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":361,"threads_started":1}
I20260807 19:44:31.017465 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:30.995043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:35425)
0807 19:44:30.996054 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:30.996112 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:30.998084 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:30.998971 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:30.998979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:30.999755 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.000519 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.000539 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.002139 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.002143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.003525 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.003539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.004384 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.004389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.004601 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.005373 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.005403 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.008034 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.012047 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.012058 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.012079 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.012764 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.013436 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.013441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.013444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.013798 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.014602 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.014609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.015186 (+ 577us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.015517 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.016012 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":437,"threads_started":1}
I20260807 19:44:31.019302 7820 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:31.019798 7820 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:31.020790 7820 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:31.022590 7342 ts_manager.cc:194] Registered new tserver with Master: e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:31.023702 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:35425
I20260807 19:44:31.027745 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.198.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.253.keytab.
Entry for principal oryx/127.0.198.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.253.keytab.
I20260807 19:44:31.097146 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:34353
--webserver_interface=127.0.198.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.253.keytab
--principal=oryx/127.0.198.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39287,127.0.198.253:34353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:31.457763 7680 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
W20260807 19:44:31.466992 7826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:31.467361 7826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:31.467615 7826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:31.475288 7826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:31.475476 7826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:31.475584 7826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:31.475688 7826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:31.475798 7826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:31.486789 7826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39287,127.0.198.253:34353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.253.keytab
--principal=oryx/127.0.198.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:34353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.198.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
W20260807 19:44:31.487237 7826 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.198.254:39287,127.0.198.253:34353'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 19:44:31.488466 7826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:31.489923 7826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:31.503141 7833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:31.503185 7835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:31.503250 7832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:31.504349 7826 server_base.cc:1034] running on GCE node
I20260807 19:44:31.505280 7826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:31.506892 7826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:31.508137 7826 hybrid_clock.cc:648] HybridClock initialized: now 1786131871508077 us; error 36 us; skew 500 ppm
I20260807 19:44:31.508533 7826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:31.512173 7826 init.cc:377] Logged in from keytab as oryx/127.0.198.253@KRBTEST.COM (short username oryx)
I20260807 19:44:31.513953 7826 webserver.cc:460] Webserver started at http://127.0.198.253:32951/ using document root <none> and password file <none>
I20260807 19:44:31.514703 7826 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:31.514804 7826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:31.515103 7826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:31.518873 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/instance:
uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
format_stamp: "Formatted at 2026-08-07 19:44:31 on dist-test-slave-btw8"
server_key: "c8178ba140235ea4406ae3b1c86cb81a"
server_key_iv: "5a815f48c3dcc6a6ad62acc912e01b3a"
server_key_version: "encryptionkey@0"
I20260807 19:44:31.519968 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal/instance:
uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
format_stamp: "Formatted at 2026-08-07 19:44:31 on dist-test-slave-btw8"
server_key: "c8178ba140235ea4406ae3b1c86cb81a"
server_key_iv: "5a815f48c3dcc6a6ad62acc912e01b3a"
server_key_version: "encryptionkey@0"
I20260807 19:44:31.526340 7826 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:31.530282 7842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:31.531976 7826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:31.532202 7826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
format_stamp: "Formatted at 2026-08-07 19:44:31 on dist-test-slave-btw8"
server_key: "c8178ba140235ea4406ae3b1c86cb81a"
server_key_iv: "5a815f48c3dcc6a6ad62acc912e01b3a"
server_key_version: "encryptionkey@0"
I20260807 19:44:31.532431 7826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:31.558844 7826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:31.563889 7826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:31.564214 7826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:31.586133 7826 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.253:34353
I20260807 19:44:31.586155 7895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.253:34353 every 8 connection(s)
I20260807 19:44:31.587803 7826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
I20260807 19:44:31.592098 7896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:31.597851 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7826
I20260807 19:44:31.600122 7896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.253@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
I20260807 19:44:31.623102 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.603313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.0.198.254:39287)
0807 19:44:31.603903 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.603912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.604543 (+ 631us) server_negotiation.cc:407] Connection header received
0807 19:44:31.605083 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.605088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.605175 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.605388 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.606719 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.607722 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.609699 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.610379 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.610477 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.615178 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.615184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.615187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.615232 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.618937 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.620189 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.620195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.620199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.620378 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.620970 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.620974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.620977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.621390 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.621566 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.622435 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.622695 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":240,"threads_started":1}
I20260807 19:44:31.624487 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.603409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55392)
0807 19:44:31.604359 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.604415 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.604785 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.605468 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.605475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.606109 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.606511 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.606528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.607867 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.607874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.609555 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.609567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.610607 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.610618 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.610929 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.611878 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.611901 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.614850 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.619177 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.619190 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.619219 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.620012 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.620548 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.620552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.620555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.620817 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.621538 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.621547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.622027 (+ 480us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.622332 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.622986 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":1,"spinlock_wait_cycles":25984,"thread_start_us":481,"threads_started":1}
I20260807 19:44:31.624506 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.600712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.0.198.253:34353)
0807 19:44:31.601695 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.601711 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.601764 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:44:31.604411 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.604443 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.604849 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.605083 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.607078 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.608712 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.610325 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.611142 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.611329 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.616976 (+ 5647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.616988 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.617004 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.617040 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.620958 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.621821 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.621831 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.621834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.621977 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.622409 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.622417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.622421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.622884 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.623235 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.623720 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.624083 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":742,"spinlock_wait_cycles":17152,"thread_start_us":486,"threads_started":1}
I20260807 19:44:31.627907 7896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.253@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
I20260807 19:44:31.642438 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.628653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44592)
0807 19:44:31.628899 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.628920 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.629022 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.629312 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.629315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.629666 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.629810 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.629817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.630831 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.630835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.631833 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.631845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.632511 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.632515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.632652 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.633371 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.633388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.635719 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.640051 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.640056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.640059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.640582 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.640998 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.641001 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.641003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.641148 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.641678 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.641681 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.641815 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.641931 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.642180 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 19:44:31.642913 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.628852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.0.198.253:34353)
0807 19:44:31.629004 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.629009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.629024 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:44:31.629115 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.629118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.629190 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.629271 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.629931 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.630667 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.631964 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.632350 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.632449 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.635937 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.635943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.635946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.635980 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.639843 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.640730 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.640734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.640736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.640886 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.641253 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.641257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.641259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.641568 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.641703 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.642110 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.642330 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:44:31.648341 7896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Bootstrap starting.
I20260807 19:44:31.651722 7896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:31.652832 7896 log.cc:824] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:31.655946 7896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: No bootstrap required, opened a new log
I20260807 19:44:31.663614 7896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } }
I20260807 19:44:31.663991 7896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:31.664058 7896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9058b5b7d85b43b4861d567af1f6f4ae, State: Initialized, Role: FOLLOWER
I20260807 19:44:31.664626 7896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } }
I20260807 19:44:31.665696 7904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } }
I20260807 19:44:31.666169 7904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:31.666764 7896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:44:31.677150 7915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:44:31.677320 7915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:44:31.680311 7826 master_runner.cc:428] Detected that this master 9058b5b7d85b43b4861d567af1f6f4ae is joining an existing cluster
I20260807 19:44:31.680429 7826 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.198.253:34353
I20260807 19:44:31.681974 7904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:31.682322 7904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } }
I20260807 19:44:31.683873 7904 leader_election.cc:288] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287)
I20260807 19:44:31.684643 7353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" is_pre_election: true
I20260807 19:44:31.686106 7844 leader_election.cc:302] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 9058b5b7d85b43b4861d567af1f6f4ae; no voters: e0c2ec4c9adf47618b38e0316eb1472f
I20260807 19:44:31.686961 7904 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:44:31.710764 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.692449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.0.198.253:34353)
0807 19:44:31.692630 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.692636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.694113 (+ 1477us) server_negotiation.cc:407] Connection header received
0807 19:44:31.694313 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.694318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.694397 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.694492 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.696173 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.697380 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.699495 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.700218 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.700317 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.703254 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.703265 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.703270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.703306 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.707099 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.708074 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.708079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.708083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.708266 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.708817 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.708822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.708825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.709313 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.709475 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.710037 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.710302 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":70400}
I20260807 19:44:31.712183 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.692115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44608)
0807 19:44:31.693977 (+ 1862us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.694027 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.694169 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.694642 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.694647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.695523 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.695992 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.696006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.697605 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.697611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.699282 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.699297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.700632 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.700638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.700905 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.701683 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.701715 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.703022 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.707267 (+ 4245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.707272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.707277 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.707945 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.708490 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.708495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.708498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.708679 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.709540 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.709545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.709713 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.711382 (+ 1669us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.711705 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1633,"mutex_wait_us":30,"thread_start_us":1416,"threads_started":1}
I20260807 19:44:31.712232 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.691216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.0.198.254:39287)
0807 19:44:31.691481 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.691488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.691749 (+ 261us) server_negotiation.cc:407] Connection header received
0807 19:44:31.692219 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.692228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.692365 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.692463 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.694348 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.695603 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.699251 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.700111 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.700269 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.703681 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.703691 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.703698 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.703749 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.708418 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.709430 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.709438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.709445 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.709629 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.710150 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.710158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.710164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.710580 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.710731 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.711483 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.711778 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 19:44:31.714601 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.690803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55400)
0807 19:44:31.691617 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.691642 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.691937 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.692740 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.692747 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.693604 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.694154 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.694169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.695837 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.695842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.699030 (+ 3188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.699052 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.700687 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.700696 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.700974 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.702037 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.702064 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.703415 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.708639 (+ 5224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.708646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.708652 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.709270 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.709795 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.709802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.709809 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.709998 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.710842 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.710848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.711070 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.712519 (+ 1449us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.712941 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":11,"thread_start_us":686,"threads_started":1}
WARNING: no policy specified for oryx/127.0.198.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.252@KRBTEST.COM" created.
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.254@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
I20260807 19:44:31.741566 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.722612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.0.198.253:34353)
0807 19:44:31.722861 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.722871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.723621 (+ 750us) server_negotiation.cc:407] Connection header received
0807 19:44:31.724047 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.724051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.724127 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.724256 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.725740 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.726469 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.727935 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.728428 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.728573 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.732987 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.732995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.733001 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.733046 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.737482 (+ 4436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.738782 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.738789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.738792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.738973 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.739680 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.739686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.739689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.740090 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.740247 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.740934 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.741193 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:44:31.741804 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.722465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44616)
0807 19:44:31.723479 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.723521 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.723770 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.724374 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.724382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.725232 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.725567 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.725580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.726628 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.726636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.727784 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.727798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.728615 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.728622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.728825 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.729587 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.729613 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.732692 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.737724 (+ 5032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.737750 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.737762 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.738581 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.739135 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.739144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.739150 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.739517 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.740259 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.740271 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.740533 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.740765 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.741264 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":595,"threads_started":1}
I20260807 19:44:31.744618 7342 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.198.253:34353: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
member_type: NON_VOTER
last_known_addr {
host: "127.0.198.253"
port: 34353
}
attrs {
promote: true
}
}
dest_uuid: "e0c2ec4c9adf47618b38e0316eb1472f"
cas_config_opid_index: -1
I20260807 19:44:31.746345 7342 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } }
I20260807 19:44:31.752300 7929 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:31.753284 7870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:31.758375 7870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 19:44:31.758373 7929 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.198.253:34353
I20260807 19:44:31.759438 7826 master.cc:537] Master@127.0.198.253:34353 shutting down...
I20260807 19:44:31.759316 7928 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:31.759728 7928 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:31.760118 7929 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.252.keytab.
Entry for principal oryx/127.0.198.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.252.keytab.
I20260807 19:44:31.764438 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:41679
--webserver_interface=127.0.198.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.252.keytab
--principal=oryx/127.0.198.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:31.765425 7327 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 19:44:31.778805 7826 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 19:44:31.779170 7327 consensus_peers.cc:487] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f -> Peer 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253:34353): Couldn't send request to peer 9058b5b7d85b43b4861d567af1f6f4ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:44:31.779567 7826 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:31.779775 7826 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: stopping tablet replica
I20260807 19:44:31.796911 7826 master.cc:552] Master@127.0.198.253:34353 shutdown complete.
I20260807 19:44:31.797084 7826 master_runner.cc:305] Clearing existing system tablet
I20260807 19:44:31.797586 7826 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:31.797710 7826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:31.800136 7826 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 19:44:31.801718 7935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:31.802073 7826 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:31.802379 7826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 19:44:31.802471 7826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
format_stamp: "Formatted at 2026-08-07 19:44:31 on dist-test-slave-btw8"
server_key: "c8178ba140235ea4406ae3b1c86cb81a"
server_key_iv: "5a815f48c3dcc6a6ad62acc912e01b3a"
server_key_version: "encryptionkey@0"
I20260807 19:44:31.806130 7826 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:31.811645 7826 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:44:31.811869 7826 log.cc:1196] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 19:44:31.812340 7826 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Deleting consensus metadata
I20260807 19:44:31.812614 7826 master_runner.cc:315] Copying system tablet from 127.0.198.254:39287
I20260807 19:44:31.816269 7826 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: tablet copy: Beginning tablet copy session from remote peer at address 127.0.198.254:39287
I20260807 19:44:31.831590 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.817378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.0.198.254:39287)
0807 19:44:31.817593 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:31.817601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:31.818142 (+ 541us) server_negotiation.cc:407] Connection header received
0807 19:44:31.818410 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:31.818420 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:31.818507 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:31.818626 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:31.819632 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.820419 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.821713 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.822314 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.822512 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.824778 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:31.824785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:31.824790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:31.824836 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:31.828426 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.829294 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.829302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.829312 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.829451 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:31.829891 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:31.829897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:31.829900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:31.830229 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:31.830400 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:31.830854 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 19:44:31.831113 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:44:31.831633 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:31.817350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55404)
0807 19:44:31.818003 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:31.818035 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:31.818224 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:31.818659 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:31.818663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:31.819107 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:31.819433 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.819447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.820604 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.820608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.821547 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:31.821559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:31.822513 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:31.822519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:31.822725 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:31.823470 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:31.823489 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:31.824466 (+ 977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:31.828638 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.828645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.828649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.829130 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.829584 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:31.829591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:31.829595 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:31.829763 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:31.830358 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:31.830363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:31.830548 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 19:44:31.830704 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 19:44:31.831107 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":408,"threads_started":1}
I20260807 19:44:31.833042 7363 tablet_copy_service.cc:140] P e0c2ec4c9adf47618b38e0316eb1472f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9058b5b7d85b43b4861d567af1f6f4ae ({username='oryx', principal='oryx/127.0.198.253@KRBTEST.COM'} at 127.0.0.1:55404)
I20260807 19:44:31.833436 7363 tablet_copy_service.cc:161] P e0c2ec4c9adf47618b38e0316eb1472f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9058b5b7d85b43b4861d567af1f6f4ae at {username='oryx', principal='oryx/127.0.198.253@KRBTEST.COM'} at 127.0.0.1:55404: session id = 9058b5b7d85b43b4861d567af1f6f4ae-00000000000000000000000000000000
I20260807 19:44:31.837215 7363 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:44:31.840557 7826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:44:31.846442 7826 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: tablet copy: Starting download of 0 data blocks...
I20260807 19:44:31.846935 7826 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: tablet copy: Starting download of 1 WAL segments...
I20260807 19:44:31.851011 7826 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:44:31.855274 7363 tablet_copy_service.cc:342] P e0c2ec4c9adf47618b38e0316eb1472f: Request end of tablet copy session 9058b5b7d85b43b4861d567af1f6f4ae-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.198.253@KRBTEST.COM'} at 127.0.0.1:55404
I20260807 19:44:31.855623 7363 tablet_copy_service.cc:433] P e0c2ec4c9adf47618b38e0316eb1472f: ending tablet copy session 9058b5b7d85b43b4861d567af1f6f4ae-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9058b5b7d85b43b4861d567af1f6f4ae
I20260807 19:44:31.868969 7826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:31.872138 7943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:31.878206 7944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:31.883541 7946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:31.885555 7826 server_base.cc:1034] running on GCE node
I20260807 19:44:31.886042 7826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:31.886806 7826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:31.888245 7826 hybrid_clock.cc:648] HybridClock initialized: now 1786131871888173 us; error 42 us; skew 500 ppm
I20260807 19:44:31.888521 7826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:31 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:31.890897 7826 init.cc:377] Logged in from keytab as oryx/127.0.198.253@KRBTEST.COM (short username oryx)
I20260807 19:44:31.892695 7826 webserver.cc:460] Webserver started at http://127.0.198.253:39979/ using document root <none> and password file <none>
I20260807 19:44:31.893095 7826 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:31.893203 7826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:31.896543 7826 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:44:31.898737 7953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:31.899253 7826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 19:44:31.899487 7826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
format_stamp: "Formatted at 2026-08-07 19:44:31 on dist-test-slave-btw8"
server_key: "c8178ba140235ea4406ae3b1c86cb81a"
server_key_iv: "5a815f48c3dcc6a6ad62acc912e01b3a"
server_key_version: "encryptionkey@0"
I20260807 19:44:31.899691 7826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:31.923485 7826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:31.928020 7826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:31.959550 7826 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.253:34353
I20260807 19:44:31.959633 8006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.253:34353 every 8 connection(s)
I20260807 19:44:31.960655 7826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
I20260807 19:44:31.967983 8007 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:44:31.970631 8007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Bootstrap starting.
I20260807 19:44:31.985534 8007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:44:31.986123 8007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: Bootstrap complete.
I20260807 19:44:31.987093 8007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } }
I20260807 19:44:31.987385 8007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9058b5b7d85b43b4861d567af1f6f4ae, State: Initialized, Role: LEARNER
I20260807 19:44:31.987632 8007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } }
I20260807 19:44:31.988572 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:31.988891 8007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:31.989009 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: LEARNER
I20260807 19:44:32.012259 8022 catalog_manager.cc:1259] Loaded cluster ID: a83fcd82a43c4b6a89bd1f0904f1191f
I20260807 19:44:32.012389 8022 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: loading cluster ID for follower catalog manager: success
I20260807 19:44:32.016547 8022 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: acquiring CA information for follower catalog manager: success
I20260807 19:44:32.020059 8022 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:44:32.026889 7820 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
I20260807 19:44:32.202991 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.193517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44624)
0807 19:44:32.193847 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.193870 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.194044 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.195363 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.195370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.195407 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:32.195639 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.195653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.197801 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.197808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.200028 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.200051 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.202092 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.202099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.202363 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.202439 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.202543 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.202701 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260807 19:44:32.203469 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.193821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.0.198.253:34353)
0807 19:44:32.194864 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.194870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.194895 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:44:32.194974 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.194980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.195073 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.195331 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:32.195890 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.197602 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.200312 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.201870 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.202012 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.202288 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.202623 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.202771 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":591,"threads_started":1}
I20260807 19:44:32.204907 7981 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 19:44:32.205647 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c2ec4c9adf47618b38e0316eb1472f. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:32.205958 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: LEARNER
W20260807 19:44:32.219614 7933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:32.219955 7933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:32.220155 7933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:32.228246 7933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:32.228382 7933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:32.228475 7933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:32.228587 7933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:32.228675 7933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:32.239821 7933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.252.keytab
--principal=oryx/127.0.198.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:41679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.198.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:32.241326 7933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:32.242759 7933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:32.256664 8028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:32.256752 8031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:32.256816 8029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:32.261197 7933 server_base.cc:1034] running on GCE node
I20260807 19:44:32.262269 7933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:32.263937 7933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:32.265208 7933 hybrid_clock.cc:648] HybridClock initialized: now 1786131872265143 us; error 45 us; skew 500 ppm
I20260807 19:44:32.265674 7933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:32.269532 7933 init.cc:377] Logged in from keytab as oryx/127.0.198.252@KRBTEST.COM (short username oryx)
I20260807 19:44:32.271562 7933 webserver.cc:460] Webserver started at http://127.0.198.252:43613/ using document root <none> and password file <none>
I20260807 19:44:32.272347 7933 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:32.272490 7933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:32.272837 7933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:32.276172 7933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/instance:
uuid: "071b89d70dfa4f91a348181543a4ba0c"
format_stamp: "Formatted at 2026-08-07 19:44:32 on dist-test-slave-btw8"
server_key: "6d021217cea153f529f077f6dff733a6"
server_key_iv: "d25ee17f38a45d3313f7cd5cb6b7b856"
server_key_version: "encryptionkey@0"
I20260807 19:44:32.277096 7933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal/instance:
uuid: "071b89d70dfa4f91a348181543a4ba0c"
format_stamp: "Formatted at 2026-08-07 19:44:32 on dist-test-slave-btw8"
server_key: "6d021217cea153f529f077f6dff733a6"
server_key_iv: "d25ee17f38a45d3313f7cd5cb6b7b856"
server_key_version: "encryptionkey@0"
I20260807 19:44:32.283094 7933 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:44:32.286993 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:32.288769 7933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:32.289022 7933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "071b89d70dfa4f91a348181543a4ba0c"
format_stamp: "Formatted at 2026-08-07 19:44:32 on dist-test-slave-btw8"
server_key: "6d021217cea153f529f077f6dff733a6"
server_key_iv: "d25ee17f38a45d3313f7cd5cb6b7b856"
server_key_version: "encryptionkey@0"
I20260807 19:44:32.289253 7933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:32.313906 7933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:32.319581 7933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:32.319926 7933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:32.341559 7933 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.252:41679
I20260807 19:44:32.341593 8095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.252:41679 every 8 connection(s)
I20260807 19:44:32.343163 7933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
I20260807 19:44:32.347640 8096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:32.349329 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 7933
I20260807 19:44:32.358073 8096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.252@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:32.379674 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.352079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58544 (local address 127.0.198.252:41679)
0807 19:44:32.353113 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.353129 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.353215 (+ 86us) server_negotiation.cc:407] Connection header received
0807 19:44:32.355169 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.355202 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.355846 (+ 644us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.356371 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.358225 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.360188 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.362543 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.363569 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.363709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.369480 (+ 5771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.369492 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.369508 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.369553 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.373798 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.374826 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.374839 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.374851 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.375116 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.375647 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.375652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.375654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.376563 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.376814 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.377548 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.378046 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":469,"threads_started":1}
I20260807 19:44:32.381562 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.361148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55414)
0807 19:44:32.361944 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.361998 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.362155 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.362987 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.362993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.364051 (+ 1058us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.364477 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.364489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.365707 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.365716 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.367849 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.367865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.368880 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.368889 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.369074 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.370032 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.370061 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.373589 (+ 3528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.378091 (+ 4502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.378100 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.378108 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.378797 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.379348 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.379371 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.379375 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.379553 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.380303 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.380310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.380526 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.380749 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.381133 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":360,"threads_started":1}
I20260807 19:44:32.381582 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.361314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.0.198.254:39287)
0807 19:44:32.362602 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.362608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.362628 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:44:32.362687 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.362692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.362779 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.362894 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.364694 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.365490 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.368041 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.368587 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.368752 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.373900 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.373908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.373914 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.373965 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.377953 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.378998 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.379004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.379007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.379210 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.379723 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.379730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.379733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.380160 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.380292 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.380912 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.381120 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1163,"thread_start_us":1031,"threads_started":1}
I20260807 19:44:32.383144 8096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.252@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
I20260807 19:44:32.398301 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.383982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44626)
0807 19:44:32.384203 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.384224 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.384331 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.384768 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.384771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.385120 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.385268 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.385277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.386372 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.386376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.387342 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.387431 (+ 89us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.388292 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.388298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.388457 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.389346 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.389365 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.391847 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.395562 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.395568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.395573 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.396086 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.396546 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.396551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.396555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.396711 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.397262 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.397267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.397427 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.397573 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.397907 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:44:32.398341 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.384181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.0.198.253:34353)
0807 19:44:32.384393 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.384400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.384442 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:44:32.384505 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.384510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.384609 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.384742 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.385433 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.386215 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.387577 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.388043 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.388164 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.392114 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.392121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.392124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.392163 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.395328 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.396263 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.396270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.396273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.396422 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.396827 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.396833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.396836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.397139 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.397300 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.397690 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.397933 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:44:32.399605 8096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.252@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
I20260807 19:44:32.413817 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.400590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.0.1:58558)
0807 19:44:32.400773 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.400793 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.400898 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.401215 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.401219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.401694 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.401845 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.401853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.402863 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.402868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.403989 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.404003 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.404751 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.404756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.404932 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.405579 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.405596 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.407872 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.411297 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.411301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.411304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.411788 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.412205 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.412210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.412213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.412384 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.412955 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.412959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.413120 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.413241 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.413451 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:44:32.413864 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.400514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.0.198.252:41679)
0807 19:44:32.400749 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.400755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.400855 (+ 100us) server_negotiation.cc:407] Connection header received
0807 19:44:32.400975 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.400978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.401036 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.401115 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.402005 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.402711 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.404159 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.404573 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.404688 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.408121 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.408127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.408130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.408163 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.411113 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.411907 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.411911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.411914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.412034 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.412502 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.412505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.412507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.412827 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.412945 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.413423 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.413608 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:44:32.420042 8096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Bootstrap starting.
I20260807 19:44:32.424235 8096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:32.425567 8096 log.cc:824] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:32.429006 8096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: No bootstrap required, opened a new log
I20260807 19:44:32.438345 8096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } }
I20260807 19:44:32.438817 8096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:32.438973 8096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071b89d70dfa4f91a348181543a4ba0c, State: Initialized, Role: FOLLOWER
I20260807 19:44:32.439790 8096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } }
I20260807 19:44:32.440811 8103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } } }
I20260807 19:44:32.441149 8103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:32.442067 8096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:44:32.452862 8114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:44:32.453032 8114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:44:32.458309 7933 master_runner.cc:428] Detected that this master 071b89d70dfa4f91a348181543a4ba0c is joining an existing cluster
I20260807 19:44:32.458482 7933 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.198.252:41679
I20260807 19:44:32.487635 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7406
I20260807 19:44:32.488973 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.465399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.0.198.254:39287)
0807 19:44:32.465613 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.465623 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.468095 (+ 2472us) server_negotiation.cc:407] Connection header received
0807 19:44:32.468453 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.468459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.468580 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.468671 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.470637 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.472023 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.474596 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.475410 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.475535 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.480867 (+ 5332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.480877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.480884 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.480934 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.485060 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.486024 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.486029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.486032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.486219 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.486768 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.486773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.486777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.487170 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.487310 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.488321 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.488598 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:44:32.490801 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.464867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55430)
0807 19:44:32.467946 (+ 3079us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.467972 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.468240 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.468959 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.468964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.469962 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.470403 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.470421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.472287 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.472293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.474366 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.474387 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.475932 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.475940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.476157 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.477086 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.477121 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.479838 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.485265 (+ 5427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.485275 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.485279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.485858 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.486381 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.486385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.486388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.486607 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.487432 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.487438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.487868 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.489592 (+ 1724us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.490057 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2909,"thread_start_us":2735,"threads_started":1}
I20260807 19:44:32.509280 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.464850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.0.198.252:41679)
0807 19:44:32.465115 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.465123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.478755 (+ 13632us) server_negotiation.cc:407] Connection header received
0807 19:44:32.479406 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.479416 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.479510 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.479592 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.481362 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.482708 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.485002 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.485826 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.485943 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.489108 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.489115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.489120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.489159 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.493652 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.494672 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.494678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.494681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.494842 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.495277 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.495288 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.495292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.497311 (+ 2019us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.497461 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.506866 (+ 9405us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.507416 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":248,"server-negotiator.queue_time_us":74}
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.193@KRBTEST.COM
I20260807 19:44:32.511250 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.465147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.0.1:58560)
0807 19:44:32.478547 (+ 13400us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.478582 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.479170 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.479823 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.479828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.480657 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.481170 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.481186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.482957 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.482962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.484806 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.484822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.486323 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.486331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.486566 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.487390 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.487416 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.488887 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.493856 (+ 4969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.493861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.493866 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.494536 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.494989 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.494994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.494997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.495138 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.505555 (+ 10417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.505562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.505831 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.506723 (+ 892us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.508112 (+ 1389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13101,"thread_start_us":12783,"threads_started":1}
I20260807 19:44:32.513464 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.468638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.0.198.253:34353)
0807 19:44:32.468903 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.468911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.480545 (+ 11634us) server_negotiation.cc:407] Connection header received
0807 19:44:32.480727 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.480734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.480818 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.480913 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.482722 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.483926 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.496852 (+ 12926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.497618 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.497744 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.501358 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.501365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.501369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.501420 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.505718 (+ 4298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.510694 (+ 4976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.510701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.510704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.510924 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.511532 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.511538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.511542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.511968 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.512134 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.512796 (+ 662us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.513081 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:44:32.514582 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.469109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:44634)
0807 19:44:32.480408 (+ 11299us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.480425 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.480577 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.481124 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.481128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.481997 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.482513 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.482529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.484130 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.484135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.496401 (+ 12266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.496419 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.498212 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.498219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.498880 (+ 661us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.499725 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.499748 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.501073 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.506686 (+ 5613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.506692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.506695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.510491 (+ 3796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.511106 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.511111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.511114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.511377 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.512221 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.512255 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.512483 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.513732 (+ 1249us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.514106 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11153,"thread_start_us":10920,"threads_started":1}
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.254@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
I20260807 19:44:32.523932 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.489615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.0.198.193:32957)
0807 19:44:32.496343 (+ 6728us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.496354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.496387 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:44:32.496466 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.496478 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.496788 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.498978 (+ 2190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.499272 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.501134 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.503673 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.505002 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.505969 (+ 967us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.514191 (+ 8222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.514217 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.514238 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.514311 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.519104 (+ 4793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.520113 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.520124 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.520128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.520306 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.520842 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.520848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.520852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.521379 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.521732 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.522474 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.523409 (+ 935us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6510,"thread_start_us":6908,"threads_started":1}
I20260807 19:44:32.525668 7535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:32.529404 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.498710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.0.1:58570)
0807 19:44:32.499096 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.499115 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.499236 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.507996 (+ 8760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.508003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.508572 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.508871 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.508890 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.512859 (+ 3969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.512866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.514252 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.514277 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.516148 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.516156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.516365 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.517258 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.517286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.520400 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.525058 (+ 4658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.525066 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.525071 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.525693 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.526345 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.526354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.526361 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.526605 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.528234 (+ 1629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.528240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.528451 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.528628 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.528983 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260807 19:44:32.530963 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.499501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.0.198.252:41679)
0807 19:44:32.507178 (+ 7677us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.507184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.507208 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:44:32.507331 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.507340 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.507527 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.509097 (+ 1570us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.509925 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.512640 (+ 2715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.514945 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.515916 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.516684 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.520727 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.520738 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.520745 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.520802 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.524846 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.525870 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.525878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.525885 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.526114 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.526869 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.526876 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.526882 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.528000 (+ 1118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.528280 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.530245 (+ 1965us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.530538 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7511,"thread_start_us":6889,"threads_started":1}
I20260807 19:44:32.532970 7342 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.198.252:41679: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c"
member_type: NON_VOTER
last_known_addr {
host: "127.0.198.252"
port: 41679
}
attrs {
promote: true
}
}
dest_uuid: "e0c2ec4c9adf47618b38e0316eb1472f"
cas_config_opid_index: 5
I20260807 19:44:32.534134 7342 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
I20260807 19:44:32.536041 7928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.537000 8068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:32.537606 7981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:44:32.538316 8127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:44:32.540779 8068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:44:32.546800 8130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:32.551455 7928 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.198.252:41679
I20260807 19:44:32.551496 7981 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.552227 7933 master.cc:537] Master@127.0.198.252:41679 shutting down...
I20260807 19:44:32.553382 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.553748 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:32.556746 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.557125 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: LEARNER
I20260807 19:44:32.558427 8130 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: attempting to promote NON_VOTER 9058b5b7d85b43b4861d567af1f6f4ae to VOTER
I20260807 19:44:32.559381 8130 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
I20260807 19:44:32.566062 7981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 LEARNER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 19:44:32.566972 7928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:44:32.569619 8068 raft_consensus.cc:858] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
I20260807 19:44:32.570003 7928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.570703 7981 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.574456 7928 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.574851 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.575105 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:32.575196 7928 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:32.574942 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.575400 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
W20260807 19:44:32.581023 7326 consensus_peers.cc:487] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f -> Peer 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252:41679): Couldn't send request to peer 071b89d70dfa4f91a348181543a4ba0c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
W20260807 19:44:32.569873 8068 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
W20260807 19:44:32.591738 8044 tablet.cc:2369] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:32.601428 8068 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:32.606387 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:32.607002 8103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c2ec4c9adf47618b38e0316eb1472f. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:32.607228 8103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 19:44:32.607609 8136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: NON_VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: true } } }
I20260807 19:44:32.607980 8136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 19:44:32.608161 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 19:44:32.610383 7933 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 19:44:32.610945 7933 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 19:44:32.611155 7933 tablet_replica.cc:330] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: stopping tablet replica
I20260807 19:44:32.620328 7933 master.cc:552] Master@127.0.198.252:41679 shutdown complete.
I20260807 19:44:32.620445 7933 master_runner.cc:305] Clearing existing system tablet
I20260807 19:44:32.620867 7933 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:32.620971 7933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:32.624293 7933 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260807 19:44:32.626109 8141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:32.626492 7933 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:32.626760 7933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 19:44:32.626858 7933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "071b89d70dfa4f91a348181543a4ba0c"
format_stamp: "Formatted at 2026-08-07 19:44:32 on dist-test-slave-btw8"
server_key: "6d021217cea153f529f077f6dff733a6"
server_key_iv: "d25ee17f38a45d3313f7cd5cb6b7b856"
server_key_version: "encryptionkey@0"
I20260807 19:44:32.632133 7933 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:32.640692 7933 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:44:32.640913 7933 log.cc:1196] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 19:44:32.641508 7933 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Deleting consensus metadata
I20260807 19:44:32.641819 7933 master_runner.cc:315] Copying system tablet from 127.0.198.254:39287
I20260807 19:44:32.645287 7933 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.198.254:39287
I20260807 19:44:32.667671 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.646302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.0.198.254:39287)
0807 19:44:32.646553 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.646560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.652143 (+ 5583us) server_negotiation.cc:407] Connection header received
0807 19:44:32.652362 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.652369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.652471 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.652614 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:32.653786 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.654980 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.656884 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.657565 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.657706 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.660381 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:32.660389 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:32.660394 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:32.660442 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:32.664213 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.665149 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.665156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.665162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.665346 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:32.665834 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:32.665840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:32.665843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:32.666288 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:32.666452 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.666968 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.667229 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 19:44:32.667695 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.646192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:55446)
0807 19:44:32.651903 (+ 5711us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.651929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.652179 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.652665 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.652677 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.653280 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:32.653567 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.653581 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.655176 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.655182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.656702 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.656721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.657764 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.657770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.657982 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.658825 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:32.658848 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:32.660132 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:32.664405 (+ 4273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.664411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.664415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.664986 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.665477 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:32.665483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:32.665487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:32.665699 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:32.666436 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:32.666441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:32.666651 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.666811 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.667181 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5540,"thread_start_us":5367,"threads_started":1}
I20260807 19:44:32.668875 7363 tablet_copy_service.cc:140] P e0c2ec4c9adf47618b38e0316eb1472f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 071b89d70dfa4f91a348181543a4ba0c ({username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:55446)
I20260807 19:44:32.669067 7363 tablet_copy_service.cc:161] P e0c2ec4c9adf47618b38e0316eb1472f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 071b89d70dfa4f91a348181543a4ba0c at {username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:55446: session id = 071b89d70dfa4f91a348181543a4ba0c-00000000000000000000000000000000
I20260807 19:44:32.673224 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7406
I20260807 19:44:32.676191 7363 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:44:32.678438 7933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:44:32.685883 7933 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: tablet copy: Starting download of 0 data blocks...
I20260807 19:44:32.686241 7933 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: tablet copy: Starting download of 1 WAL segments...
I20260807 19:44:32.689348 7933 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:44:32.694829 7363 tablet_copy_service.cc:342] P e0c2ec4c9adf47618b38e0316eb1472f: Request end of tablet copy session 071b89d70dfa4f91a348181543a4ba0c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:55446
I20260807 19:44:32.695008 7363 tablet_copy_service.cc:433] P e0c2ec4c9adf47618b38e0316eb1472f: ending tablet copy session 071b89d70dfa4f91a348181543a4ba0c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 071b89d70dfa4f91a348181543a4ba0c
I20260807 19:44:32.695719 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:32957
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:39287
--webserver_port=37087
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:32.707798 7933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:32.711066 8149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:32.717348 8150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:32.727850 7933 server_base.cc:1034] running on GCE node
W20260807 19:44:32.731690 8152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:32.732370 7933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:32.733230 7933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:32.734481 7933 hybrid_clock.cc:648] HybridClock initialized: now 1786131872734431 us; error 42 us; skew 500 ppm
I20260807 19:44:32.734787 7933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:32 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:32.737267 7933 init.cc:377] Logged in from keytab as oryx/127.0.198.252@KRBTEST.COM (short username oryx)
I20260807 19:44:32.740548 7933 webserver.cc:460] Webserver started at http://127.0.198.252:46069/ using document root <none> and password file <none>
I20260807 19:44:32.740926 7933 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:32.741066 7933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:32.745190 7933 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:44:32.747709 8160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:32.748214 7933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 19:44:32.748373 7933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "071b89d70dfa4f91a348181543a4ba0c"
format_stamp: "Formatted at 2026-08-07 19:44:32 on dist-test-slave-btw8"
server_key: "6d021217cea153f529f077f6dff733a6"
server_key_iv: "d25ee17f38a45d3313f7cd5cb6b7b856"
server_key_version: "encryptionkey@0"
I20260807 19:44:32.748600 7933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:32.766741 7933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:32.771819 7933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:32.798547 8216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.252:41679 every 8 connection(s)
I20260807 19:44:32.799154 7933 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.252:41679
I20260807 19:44:32.801582 7933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
I20260807 19:44:32.817056 8217 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:44:32.822130 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Bootstrap starting.
I20260807 19:44:32.839300 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:44:32.839917 8217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Bootstrap complete.
I20260807 19:44:32.841102 8217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
I20260807 19:44:32.841401 8217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 071b89d70dfa4f91a348181543a4ba0c, State: Initialized, Role: LEARNER
I20260807 19:44:32.841673 8217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } }
I20260807 19:44:32.842594 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.842932 8217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:32.843102 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: LEARNER
I20260807 19:44:32.852340 8231 catalog_manager.cc:1259] Loaded cluster ID: a83fcd82a43c4b6a89bd1f0904f1191f
I20260807 19:44:32.852505 8231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: loading cluster ID for follower catalog manager: success
I20260807 19:44:32.855049 8231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: acquiring CA information for follower catalog manager: success
I20260807 19:44:32.857098 8231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:44:32.993535 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.983895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.0.1:58574)
0807 19:44:32.984232 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:32.984255 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:32.984417 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:32.985767 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:32.985775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:32.985817 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:32.986041 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.986053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.988131 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.988138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.990374 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:32.990404 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.992693 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.992700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:32.992932 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.992975 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 19:44:32.993055 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:44:32.993229 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260807 19:44:32.994657 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:32.984084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.0.198.252:41679)
0807 19:44:32.985279 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:32.985292 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:32.985320 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:44:32.985418 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:32.985425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:32.985533 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:32.985687 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:32.986227 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.987911 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.990655 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:32.992414 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:32.993465 (+ 1051us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:32.994043 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:32.994114 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 19:44:32.994282 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":596,"threads_started":1}
I20260807 19:44:32.995757 8188 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 19:44:32.996692 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c2ec4c9adf47618b38e0316eb1472f. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:32.997013 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: LEARNER
W20260807 19:44:33.153918 8148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:33.154310 8148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:33.154496 8148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:33.162638 8148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:33.162817 8148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:33.162999 8148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:33.174088 8148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:32957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=37087
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:33.175822 8148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:33.177336 8148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:33.192330 8237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:33.192512 8238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:33.192698 8240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:33.193524 8148 server_base.cc:1034] running on GCE node
I20260807 19:44:33.194303 8148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:33.195469 8148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:33.196789 8148 hybrid_clock.cc:648] HybridClock initialized: now 1786131873196682 us; error 59 us; skew 500 ppm
I20260807 19:44:33.197250 8148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:33.201215 8148 init.cc:377] Logged in from keytab as oryx/127.0.198.193@KRBTEST.COM (short username oryx)
I20260807 19:44:33.202927 8148 webserver.cc:460] Webserver started at http://127.0.198.193:37087/ using document root <none> and password file <none>
I20260807 19:44:33.203797 8148 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:33.203984 8148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:33.210940 8148 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:44:33.214526 8247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:33.216488 8148 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:33.216713 8148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "cadd11c386ff4d86872dd35e4e69414b"
format_stamp: "Formatted at 2026-08-07 19:44:29 on dist-test-slave-btw8"
server_key: "6801ba7139ab71ef50a3d07140bb6a0a"
server_key_iv: "d1f597f8188c78ae0ff0e858105bfbdd"
server_key_version: "encryptionkey@0"
I20260807 19:44:33.217926 8148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:33.243549 8148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:33.249512 8148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:33.249858 8148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:33.251116 8148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:33.252871 8148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:33.253010 8148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:33.253146 8148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:33.253237 8148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:33.293059 8148 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:32957
I20260807 19:44:33.293164 8373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:32957 every 8 connection(s)
I20260807 19:44:33.294705 8148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:33.299916 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 8148
I20260807 19:44:33.300493 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7546
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.194@KRBTEST.COM
I20260807 19:44:33.334424 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.298431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:47661 (local address 127.0.198.252:41679)
0807 19:44:33.298667 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:33.298677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:33.308876 (+ 10199us) server_negotiation.cc:407] Connection header received
0807 19:44:33.309701 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:33.309711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:33.309831 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:33.309929 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:33.316008 (+ 6079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.317055 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.319391 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.320087 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.320237 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.326378 (+ 6141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:33.326386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:33.326391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:33.326443 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:33.330027 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.331494 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.331502 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.331508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.331721 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.332399 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.332407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.332410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.332735 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:33.332841 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:33.333779 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 19:44:33.334059 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:44:33.335939 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.299171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.198.193:47661)
0807 19:44:33.308661 (+ 9490us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:33.308729 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:33.309261 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:33.310280 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:33.310287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:33.312631 (+ 2344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:33.313082 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.313099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.317423 (+ 4324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.317431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.319171 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.319185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.320479 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.320485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.320676 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.322653 (+ 1977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:33.322690 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:33.326067 (+ 3377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:33.330247 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.330259 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.330283 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.331244 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.331881 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.331888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.331894 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.332224 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.332940 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:33.332947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:33.333351 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 19:44:33.333670 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 19:44:33.334282 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8574,"mutex_wait_us":62,"spinlock_wait_cycles":86784,"thread_start_us":17694,"threads_started":1}
I20260807 19:44:33.336585 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.299169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:59939)
0807 19:44:33.300595 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:33.300667 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:33.304107 (+ 3440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:33.308608 (+ 4501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:33.308628 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:33.309843 (+ 1215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:33.311846 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.311895 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.313520 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.313528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.316437 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.316464 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.318975 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.318988 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.319498 (+ 510us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.320940 (+ 1442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:33.320981 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:33.324958 (+ 3977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:33.330631 (+ 5673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.330636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.330640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.331384 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.332150 (+ 766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.332154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.332158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.332318 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.335079 (+ 2761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:33.335084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:33.335293 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 19:44:33.335482 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 19:44:33.335836 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":59,"thread_start_us":459,"threads_started":1}
I20260807 19:44:33.336875 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.298428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:59939 (local address 127.0.198.254:39287)
0807 19:44:33.299056 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:33.299063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:33.307503 (+ 8440us) server_negotiation.cc:407] Connection header received
0807 19:44:33.307628 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:33.307637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:33.307781 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:33.307932 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:33.312164 (+ 4232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.313320 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.316635 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.317424 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.317555 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.325364 (+ 7809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:33.325372 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:33.325379 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:33.325433 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:33.330400 (+ 4967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.331820 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.331827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.331833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.332037 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.334332 (+ 2295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.334337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.334340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.334773 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:33.334974 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:33.336003 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0807 19:44:33.336455 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":300,"threads_started":1}
I20260807 19:44:33.338379 8374 heartbeater.cc:347] Connected to a master server at 127.0.198.252:41679
I20260807 19:44:33.338398 8375 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:33.338788 8375 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:33.338792 8374 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:33.340077 8374 heartbeater.cc:511] Master 127.0.198.252:41679 requested a full tablet report, sending...
I20260807 19:44:33.340103 8375 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:33.340647 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.301368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.198.193:56991)
0807 19:44:33.316699 (+ 15331us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:33.316718 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:33.316947 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:33.317594 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:33.317598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:33.318338 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:33.318677 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.318691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.320861 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.320869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.322235 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:33.322248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.323392 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.323400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:33.323616 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.325282 (+ 1666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:33.325311 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:33.328292 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:33.335055 (+ 6763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.335062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.335068 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.335708 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.337928 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:33.337935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:33.337939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:33.338185 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:33.339299 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:33.339305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:33.339517 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 19:44:33.339803 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 19:44:33.340177 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15183,"thread_start_us":6202,"threads_started":1}
I20260807 19:44:33.341769 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.302295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:56991 (local address 127.0.198.253:34353)
0807 19:44:33.316192 (+ 13897us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:33.316199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:33.316837 (+ 638us) server_negotiation.cc:407] Connection header received
0807 19:44:33.317130 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:33.317137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:33.317251 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:33.317365 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:33.319206 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.320651 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.322431 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.323132 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.323293 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.328643 (+ 5350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:33.328652 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:33.328657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:33.328723 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:33.334094 (+ 5371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.337530 (+ 3436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.337537 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.337540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.337756 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.338494 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.338499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.338502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.338935 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:33.339196 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:33.341060 (+ 1864us) server_negotiation.cc:299] Negotiation successful
0807 19:44:33.341337 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13618,"thread_start_us":13749,"threads_started":1}
I20260807 19:44:33.342319 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:33.302292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.0.198.194:45393)
0807 19:44:33.311424 (+ 9132us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:33.311439 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:33.312404 (+ 965us) server_negotiation.cc:407] Connection header received
0807 19:44:33.312496 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:33.312506 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:33.312805 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:33.312987 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:33.314961 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.317608 (+ 2647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.319035 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:33.320021 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:33.320158 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:33.330994 (+ 10836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:33.331026 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:33.331049 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:33.331110 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:33.335505 (+ 4395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.337518 (+ 2013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.337529 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.337532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.338468 (+ 936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:33.339088 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:33.339094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:33.339183 (+ 89us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:33.339722 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:33.340030 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:33.341453 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0807 19:44:33.341808 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8863,"thread_start_us":8653,"threads_started":1}
I20260807 19:44:33.342707 8376 heartbeater.cc:347] Connected to a master server at 127.0.198.253:34353
I20260807 19:44:33.342865 8376 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:33.342859 8177 ts_manager.cc:194] Registered new tserver with Master: cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957)
I20260807 19:44:33.343441 8376 heartbeater.cc:511] Master 127.0.198.253:34353 requested a full tablet report, sending...
I20260807 19:44:33.343911 7675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:33.345444 7971 ts_manager.cc:194] Registered new tserver with Master: cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957)
I20260807 19:44:33.345599 7342 ts_manager.cc:194] Re-registered known tserver with Master: cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957)
I20260807 19:44:33.346812 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:59939
I20260807 19:44:33.431306 8130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: NON_VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: true } } }
I20260807 19:44:33.431951 8130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:33.433553 8130 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: attempting to promote NON_VOTER 071b89d70dfa4f91a348181543a4ba0c to VOTER
I20260807 19:44:33.434239 8130 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } }
I20260807 19:44:33.436266 8188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 LEARNER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:44:33.436285 7981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Refusing update from remote peer e0c2ec4c9adf47618b38e0316eb1472f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:44:33.436978 8388 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:44:33.437778 8387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:44:33.441372 8388 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.442160 7981 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.445573 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7546
I20260807 19:44:33.448079 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.448450 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:33.448138 8388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.448724 8388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:33.451490 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.451849 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:33.452438 8188 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.456132 8391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.456681 8391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:33.457894 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:33.458212 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:33.469573 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:45393
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:39287
--webserver_port=38361
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:33.847790 8392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:33.848100 8392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:33.848206 8392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:33.856074 8392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:33.856204 8392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:33.856329 8392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:33.866719 8392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:45393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=38361
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:33.868243 8392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:33.869642 8392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:33.885344 8397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:33.885308 8399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:33.885507 8402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:33.886219 8392 server_base.cc:1034] running on GCE node
I20260807 19:44:33.887681 8392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:33.888972 8392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:33.890228 8392 hybrid_clock.cc:648] HybridClock initialized: now 1786131873890163 us; error 65 us; skew 500 ppm
I20260807 19:44:33.890792 8392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:33 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:33.894820 8392 init.cc:377] Logged in from keytab as oryx/127.0.198.194@KRBTEST.COM (short username oryx)
I20260807 19:44:33.896600 8392 webserver.cc:460] Webserver started at http://127.0.198.194:38361/ using document root <none> and password file <none>
I20260807 19:44:33.897379 8392 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:33.897521 8392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:33.906595 8392 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:44:33.914777 8409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:33.917192 8392 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260807 19:44:33.917449 8392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "dff9d30e63e343c4818e29779a7d06a3"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "ad712aaf94d5d3b5fb6b9174dfd24d29"
server_key_iv: "72651be26878f10257063f482de86278"
server_key_version: "encryptionkey@0"
I20260807 19:44:33.918891 8392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:33.954097 8392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:33.960819 8392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:33.961126 8392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:33.962428 8392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:33.964514 8392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:33.964643 8392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:33.964757 8392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:33.964846 8392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:34.006103 8392 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:45393
I20260807 19:44:34.006141 8527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:45393 every 8 connection(s)
I20260807 19:44:34.007951 8392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:34.014308 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 8392
I20260807 19:44:34.014730 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7686
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.195@KRBTEST.COM
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:34.057435 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.011555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:52399 (local address 127.0.198.252:41679)
0807 19:44:34.023996 (+ 12441us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.024007 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.025849 (+ 1842us) server_negotiation.cc:407] Connection header received
0807 19:44:34.031225 (+ 5376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.031231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.031340 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.031457 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.036545 (+ 5088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.037839 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.041222 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.041894 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.041998 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.048906 (+ 6908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.048916 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.048925 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.048979 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.053061 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.054210 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.054226 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.054230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.054410 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.055041 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.055047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.055050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.055493 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.055655 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.056729 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.057002 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12227,"thread_start_us":12483,"threads_started":1}
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131873, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
I20260807 19:44:34.059573 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.016348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.0.198.195:42655)
0807 19:44:34.030916 (+ 14568us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.030925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.030957 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:44:34.031037 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.031047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.031377 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.031558 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.034151 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.035822 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.038368 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.039806 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.040012 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.049899 (+ 9887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.049916 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.049933 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.050008 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.054745 (+ 4737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.055876 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.055886 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.055889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.056076 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.056621 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.056626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.056629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.057065 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.057378 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.058817 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.059131 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14365,"thread_start_us":14214,"threads_started":1}
I20260807 19:44:34.060034 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.012388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.198.194:52399)
0807 19:44:34.025528 (+ 13140us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.025637 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.031041 (+ 5404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.034161 (+ 3120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.034176 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.035253 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.036282 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.036314 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.038130 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.038136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.039872 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.039888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.042391 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.042398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.042673 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.043587 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.043640 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.048544 (+ 4904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.053281 (+ 4737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.053292 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.053314 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.054033 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.054587 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.054594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.054600 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.054900 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.055630 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.055639 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.056189 (+ 550us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.057797 (+ 1608us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.058417 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12649,"thread_start_us":12334,"threads_started":1}
I20260807 19:44:34.060778 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.012388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:38423)
0807 19:44:34.026516 (+ 14128us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.026542 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.029958 (+ 3416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.040099 (+ 10141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.040107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.040722 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.040978 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.040987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.043068 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.043072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.044813 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.044832 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.045953 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.045964 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.046163 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.046927 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.046956 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.051225 (+ 4269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.056896 (+ 5671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.056901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.056905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.057493 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.058197 (+ 704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.058205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.058211 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.058380 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.059465 (+ 1085us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.059470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.059674 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.059815 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.060199 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13911,"mutex_wait_us":41,"spinlock_wait_cycles":89856,"thread_start_us":13500,"threads_started":1}
I20260807 19:44:34.061818 8528 heartbeater.cc:347] Connected to a master server at 127.0.198.252:41679
I20260807 19:44:34.061950 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.011555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:38423 (local address 127.0.198.254:39287)
0807 19:44:34.024588 (+ 13033us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.024598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.030437 (+ 5839us) server_negotiation.cc:407] Connection header received
0807 19:44:34.030551 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.030559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.030671 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.030805 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.041199 (+ 10394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.042300 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.044995 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.045714 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.045860 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.051589 (+ 5729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.051596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.051602 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.051652 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.056705 (+ 5053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.057863 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.057873 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.057880 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.058055 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.058519 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.058525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.058528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.058947 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.059116 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.061229 (+ 2113us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.061492 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12859,"thread_start_us":12734,"threads_started":1}
I20260807 19:44:34.062182 8528 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.062248 7815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:34.063383 8528 heartbeater.cc:511] Master 127.0.198.252:41679 requested a full tablet report, sending...
I20260807 19:44:34.065742 8177 ts_manager.cc:194] Registered new tserver with Master: dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393)
I20260807 19:44:34.066249 8529 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:34.066419 8529 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.067057 8529 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:34.068614 7342 ts_manager.cc:194] Re-registered known tserver with Master: dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393)
I20260807 19:44:34.069051 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.027329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:51187 (local address 127.0.198.253:34353)
0807 19:44:34.036371 (+ 9042us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.036379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.046054 (+ 9675us) server_negotiation.cc:407] Connection header received
0807 19:44:34.046138 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.046143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.046231 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.046957 (+ 726us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.049015 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.050445 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.052856 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.053638 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.053787 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.059765 (+ 5978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.059776 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.059785 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.059839 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.064712 (+ 4873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.066042 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.066051 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.066054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.066262 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.067168 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.067174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.067177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.067653 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.067801 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.068334 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.068614 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8886,"thread_start_us":9564,"threads_started":1}
I20260807 19:44:34.069068 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.031156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.198.194:51187)
0807 19:44:34.045454 (+ 14298us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.045483 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.045622 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.047379 (+ 1757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.047386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.048342 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.048837 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.048848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.050670 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.050692 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.052627 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.052649 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.054161 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.054171 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.054417 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.055975 (+ 1558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.056002 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.059457 (+ 3455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.064892 (+ 5435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.064899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.064904 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.065564 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.066736 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.066747 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.066751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.066989 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.067883 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.067886 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.068035 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.068195 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.068560 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14136,"thread_start_us":13975,"threads_started":1}
I20260807 19:44:34.069888 8530 heartbeater.cc:347] Connected to a master server at 127.0.198.253:34353
I20260807 19:44:34.070019 8530 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.070115 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:38423
I20260807 19:44:34.070405 8530 heartbeater.cc:511] Master 127.0.198.253:34353 requested a full tablet report, sending...
I20260807 19:44:34.071810 7971 ts_manager.cc:194] Registered new tserver with Master: dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393)
I20260807 19:44:34.158823 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7686
I20260807 19:44:34.178498 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:42655
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:39287
--webserver_port=36277
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:34.350337 8375 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
W20260807 19:44:34.559823 8541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:34.560227 8541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:34.560432 8541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:34.569445 8541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:34.569639 8541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:34.569842 8541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:34.581574 8541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:42655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=36277
--tserver_master_addrs=127.0.198.254:39287,127.0.198.253:34353,127.0.198.252:41679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:34.583125 8541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:34.584664 8541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:34.600124 8548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:34.600195 8549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:34.600477 8551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:34.601514 8541 server_base.cc:1034] running on GCE node
I20260807 19:44:34.602313 8541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:34.603650 8541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:34.604939 8541 hybrid_clock.cc:648] HybridClock initialized: now 1786131874604868 us; error 70 us; skew 500 ppm
I20260807 19:44:34.605327 8541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131874, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:34.609486 8541 init.cc:377] Logged in from keytab as oryx/127.0.198.195@KRBTEST.COM (short username oryx)
I20260807 19:44:34.611720 8541 webserver.cc:460] Webserver started at http://127.0.198.195:36277/ using document root <none> and password file <none>
I20260807 19:44:34.612535 8541 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:34.612726 8541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:34.619254 8541 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:44:34.623131 8558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:34.625159 8541 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:34.625417 8541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "e163a25c62af4e6e858c26742b591eca"
format_stamp: "Formatted at 2026-08-07 19:44:30 on dist-test-slave-btw8"
server_key: "641382c8d4129c6d990a58add9edff73"
server_key_iv: "b47a37e5378dbd88a4e3f08a3aea81be"
server_key_version: "encryptionkey@0"
I20260807 19:44:34.626575 8541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:34.640997 8541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:34.646054 8541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:34.646371 8541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:34.647580 8541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:34.649139 8541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:34.649283 8541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:34.649367 8541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:34.649431 8541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:34.687942 8541 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:42655
I20260807 19:44:34.688046 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:42655 every 8 connection(s)
I20260807 19:44:34.689694 8541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:34.691283 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 8541
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131874, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.253@KRBTEST.COM
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131874, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.252@KRBTEST.COM
I20260807 19:44:34.738338 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.694251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.198.195:49973)
0807 19:44:34.707836 (+ 13585us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.707863 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.708054 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.708651 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.708656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.709476 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.709938 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.709953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.711622 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.711627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.713573 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.713591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.715034 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.715043 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.715544 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.717511 (+ 1967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.717536 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.721640 (+ 4104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.728103 (+ 6463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.728122 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.728154 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.729167 (+ 1013us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.729873 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.729888 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.729895 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.730938 (+ 1043us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.733478 (+ 2540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.733497 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.734336 (+ 839us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.735013 (+ 677us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.736021 (+ 1008us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13329,"mutex_wait_us":1497,"thread_start_us":4119,"threads_started":1}
I20260807 19:44:34.741926 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.695198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:49973 (local address 127.0.198.252:41679)
0807 19:44:34.706971 (+ 11773us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.706981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.708206 (+ 1225us) server_negotiation.cc:407] Connection header received
0807 19:44:34.708312 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.708317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.708429 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.710120 (+ 1691us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.710320 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.711440 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.713776 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.714504 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.714625 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.722070 (+ 7445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.722081 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.722090 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.722154 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.727754 (+ 5600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.729435 (+ 1681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.729444 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.729451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.729681 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.732058 (+ 2377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.732098 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.732102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.733075 (+ 973us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.740613 (+ 7538us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.740887 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.741312 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11551,"thread_start_us":11266,"threads_started":1}
I20260807 19:44:34.744560 8678 heartbeater.cc:347] Connected to a master server at 127.0.198.252:41679
I20260807 19:44:34.745056 8678 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.746588 8678 heartbeater.cc:511] Master 127.0.198.252:41679 requested a full tablet report, sending...
I20260807 19:44:34.746992 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.695782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:43291 (local address 127.0.198.253:34353)
0807 19:44:34.696365 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.696376 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.700923 (+ 4547us) server_negotiation.cc:407] Connection header received
0807 19:44:34.705632 (+ 4709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.705642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.705799 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.705914 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.709510 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.710668 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.713791 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.714491 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.714614 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.736934 (+ 22320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.736944 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.736952 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.737014 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.742225 (+ 5211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.743685 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.743696 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.743702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.743968 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.744675 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.744683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.744689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.745173 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.745363 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.746076 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.746439 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":282,"threads_started":1}
I20260807 19:44:34.749548 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.694260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.198.195:43291)
0807 19:44:34.700519 (+ 6259us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.700653 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.705025 (+ 4372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.706721 (+ 1696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.706741 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.707864 (+ 1123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.709234 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.709274 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.710917 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.710923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.713573 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.713591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.715034 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.715043 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.716145 (+ 1102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.716993 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.717039 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.736586 (+ 19547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.742502 (+ 5916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.742512 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.742520 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.743259 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.744181 (+ 922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.744189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.744196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.744509 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.745470 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.745478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.745725 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.748412 (+ 2687us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.748912 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5704,"mutex_wait_us":704,"thread_start_us":12959,"threads_started":1}
I20260807 19:44:34.750633 8177 ts_manager.cc:194] Registered new tserver with Master: e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:34.750734 8680 heartbeater.cc:347] Connected to a master server at 127.0.198.253:34353
I20260807 19:44:34.750926 8680 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.751550 8680 heartbeater.cc:511] Master 127.0.198.253:34353 requested a full tablet report, sending...
I20260807 19:44:34.753010 7971 ts_manager.cc:194] Registered new tserver with Master: e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
Aug 07 19:44:34 dist-test-slave-btw8 krb5kdc[7296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131874, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:34.771287 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.708628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:44877)
0807 19:44:34.717041 (+ 8413us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.717063 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.717302 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.717967 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.717972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.734792 (+ 16820us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:34.735338 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.735381 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.751981 (+ 16600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.751986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.753599 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.753616 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.756311 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.756324 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.756676 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.757805 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:34.757838 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:34.761881 (+ 4043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:34.766906 (+ 5025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.766913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.766918 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.767895 (+ 977us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.768489 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:34.768498 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:34.768505 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:34.768710 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:34.769985 (+ 1275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:34.769993 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:34.770233 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.770390 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.770758 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8259,"thread_start_us":8275,"threads_started":1}
I20260807 19:44:34.771417 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.708771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:44877 (local address 127.0.198.254:39287)
0807 19:44:34.716151 (+ 7380us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.716158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.717173 (+ 1015us) server_negotiation.cc:407] Connection header received
0807 19:44:34.717496 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.717505 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.717606 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.717703 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.750434 (+ 32731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.751725 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.755122 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.755986 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.756156 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.762307 (+ 6151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.762316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.762324 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.762383 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.766673 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.768127 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.768131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.768134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.768327 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.769396 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.769402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.769405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.769832 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.769971 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.770624 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.770977 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7188,"thread_start_us":6918,"threads_started":1}
I20260807 19:44:34.772404 8679 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39287
I20260807 19:44:34.772653 8679 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:34.773206 8679 heartbeater.cc:511] Master 127.0.198.254:39287 requested a full tablet report, sending...
I20260807 19:44:34.774700 7342 ts_manager.cc:194] Re-registered known tserver with Master: e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:34.775954 7342 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:44877
I20260807 19:44:34.787952 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.721090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.0.198.253:34353)
0807 19:44:34.747316 (+ 26226us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.747325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.758875 (+ 11550us) server_negotiation.cc:407] Connection header received
0807 19:44:34.759241 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.759248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.759398 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.759492 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.765132 (+ 5640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.766691 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.769681 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.771096 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.771264 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.779596 (+ 8332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.779606 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.779612 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.779665 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.784104 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.785255 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.785263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.785267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.785464 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.786136 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.786142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.786145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.786611 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.786882 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.787283 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.787559 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26069,"thread_start_us":33518,"threads_started":1}
I20260807 19:44:34.789320 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.720640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.0.198.252:41679)
0807 19:44:34.754270 (+ 33630us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.754281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.756332 (+ 2051us) server_negotiation.cc:407] Connection header received
0807 19:44:34.756931 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.756940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.757077 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.757558 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.763606 (+ 6048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.764839 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.768573 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.769191 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.769307 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.780327 (+ 11020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.780714 (+ 387us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.780722 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.780781 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.784795 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.786130 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.786136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.786140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.786402 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.786918 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.786926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.786929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.787341 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.787694 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.788304 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.788949 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33395,"thread_start_us":34628,"threads_started":1}
I20260807 19:44:34.790295 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.718615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.0.198.254:39287)
0807 19:44:34.757327 (+ 38712us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.757335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.757356 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:44:34.757411 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.757419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.757508 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.757623 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:34.763295 (+ 5672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.765018 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.768013 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.768929 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.769121 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.779998 (+ 10877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:34.780006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:34.780012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:34.780055 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:34.784522 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.786315 (+ 1793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.786320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.786323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.786496 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:34.787483 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:34.787493 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:34.787497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:34.787907 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:34.788195 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.788585 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.789866 (+ 1281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38561,"thread_start_us":38512,"threads_started":1}
I20260807 19:44:34.795627 7342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:34.806384 7342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:34.844856 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.830407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.0.198.194:45393)
0807 19:44:34.836452 (+ 6045us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.836461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.836487 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:44:34.836577 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.836583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.836832 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.838002 (+ 1170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:34.838263 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.839997 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.842102 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.843612 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.843783 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.844050 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.844388 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.844564 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5873,"thread_start_us":5765,"threads_started":1}
I20260807 19:44:34.844920 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.830003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:45393 (local address 127.0.0.1:39942)
0807 19:44:34.830961 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.830983 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.831216 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.837108 (+ 5892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.837113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.837146 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:34.837745 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.837770 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.840142 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.840147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.841881 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.841903 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.843867 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.843873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.844174 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.844208 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.844353 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.844519 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":84,"thread_start_us":349,"threads_started":1}
I20260807 19:44:34.846046 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.830822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32957 (local address 127.0.0.1:37442)
0807 19:44:34.835995 (+ 5173us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.836021 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.836215 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.837346 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.837351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.837373 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:34.837985 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.838004 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.840043 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.840050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.842640 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.842656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.845161 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.845168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.845347 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.845382 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.845464 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.845643 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5015,"mutex_wait_us":34,"thread_start_us":4816,"threads_started":1}
I20260807 19:44:34.847398 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.831097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.0.198.193:32957)
0807 19:44:34.835996 (+ 4899us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.836008 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.836366 (+ 358us) server_negotiation.cc:407] Connection header received
0807 19:44:34.836497 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.836508 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.836832 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.837073 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:34.838280 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.839904 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.842990 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.844687 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.844832 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.846489 (+ 1657us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.846566 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.847041 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4660,"thread_start_us":4496,"threads_started":1}
I20260807 19:44:34.848201 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.837227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42655 (local address 127.0.0.1:34942)
0807 19:44:34.838019 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.838039 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.838267 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.839256 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.839260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.839322 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:34.839787 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.839796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.842175 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.842182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.844522 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.844538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.847041 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.847047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.847271 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.847305 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.847434 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.847801 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":6,"thread_start_us":451,"threads_started":1}
I20260807 19:44:34.848441 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.836477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.0.198.195:42655)
0807 19:44:34.837190 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.837197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.838157 (+ 960us) server_negotiation.cc:407] Connection header received
0807 19:44:34.838460 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.838469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.838757 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.838960 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:34.839986 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.841963 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.844749 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.846338 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.846442 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.846647 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.847926 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.848113 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":367,"threads_started":1}
I20260807 19:44:34.849092 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 (DEFAULT_TABLE table=test-table [id=0eabd010ee764edeb86f8316f7dcaafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:34.850493 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8def7eb90db74c7bafa0c99a662f6a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:34.851060 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 (DEFAULT_TABLE table=test-table [id=0eabd010ee764edeb86f8316f7dcaafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:34.852269 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8def7eb90db74c7bafa0c99a662f6a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:34.857861 8608 tablet_service.cc:1467] Processing CreateTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 (DEFAULT_TABLE table=test-table [id=0eabd010ee764edeb86f8316f7dcaafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:34.859505 8608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8def7eb90db74c7bafa0c99a662f6a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:34.864135 8716 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Bootstrap starting.
I20260807 19:44:34.867406 8716 tablet_bootstrap.cc:654] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:34.869434 8716 log.cc:824] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:34.870039 8717 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Bootstrap starting.
I20260807 19:44:34.873593 8716 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: No bootstrap required, opened a new log
I20260807 19:44:34.873960 8716 ts_tablet_manager.cc:1397] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:44:34.874325 8717 tablet_bootstrap.cc:654] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:34.875727 8719 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Bootstrap starting.
I20260807 19:44:34.875765 8717 log.cc:824] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:34.879977 8719 tablet_bootstrap.cc:654] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:34.880040 8717 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: No bootstrap required, opened a new log
I20260807 19:44:34.880379 8717 ts_tablet_manager.cc:1397] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260807 19:44:34.881454 8719 log.cc:824] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:34.884131 8716 raft_consensus.cc:348] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.884611 8716 raft_consensus.cc:374] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:34.884739 8716 raft_consensus.cc:729] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff9d30e63e343c4818e29779a7d06a3, State: Initialized, Role: FOLLOWER
I20260807 19:44:34.885118 8719 tablet_bootstrap.cc:492] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: No bootstrap required, opened a new log
I20260807 19:44:34.885417 8719 ts_tablet_manager.cc:1397] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:44:34.885432 8716 consensus_queue.cc:260] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.887636 8716 ts_tablet_manager.cc:1428] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260807 19:44:34.887940 8529 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
I20260807 19:44:34.893517 8717 raft_consensus.cc:348] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.894083 8717 raft_consensus.cc:374] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:34.894203 8717 raft_consensus.cc:729] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cadd11c386ff4d86872dd35e4e69414b, State: Initialized, Role: FOLLOWER
I20260807 19:44:34.894930 8717 consensus_queue.cc:260] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.896970 8717 ts_tablet_manager.cc:1428] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260807 19:44:34.898077 8719 raft_consensus.cc:348] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.898567 8719 raft_consensus.cc:374] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:34.898687 8719 raft_consensus.cc:729] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e163a25c62af4e6e858c26742b591eca, State: Initialized, Role: FOLLOWER
I20260807 19:44:34.899519 8719 consensus_queue.cc:260] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.902247 8679 heartbeater.cc:503] Master 127.0.198.254:39287 was elected leader, sending a full tablet report...
I20260807 19:44:34.902984 8719 ts_tablet_manager.cc:1428] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
W20260807 19:44:34.946077 8681 tablet.cc:2369] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:34.985074 8722 raft_consensus.cc:482] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:34.985476 8722 raft_consensus.cc:504] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:34.987429 8722 leader_election.cc:288] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957), e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:34.995307 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.987888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:48575 (local address 127.0.198.193:32957)
0807 19:44:34.988194 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.988204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.988535 (+ 331us) server_negotiation.cc:407] Connection header received
0807 19:44:34.988827 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.988834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.988937 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.989043 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:34.989968 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.990989 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.993384 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.994300 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.994511 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.994592 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.994860 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.995015 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:44:34.995484 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.987665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32957 (local address 127.0.198.194:48575)
0807 19:44:34.988406 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.988424 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.988626 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.989218 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.989222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.989250 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:34.989773 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.989788 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.991197 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.991201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.993191 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.993206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.994518 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.994524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.994692 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.994718 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.994830 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.994993 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":58,"spinlock_wait_cycles":22272,"thread_start_us":406,"threads_started":1}
I20260807 19:44:34.995870 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.988848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:42159 (local address 127.0.198.195:42655)
0807 19:44:34.989112 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:34.989118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:34.989425 (+ 307us) server_negotiation.cc:407] Connection header received
0807 19:44:34.989654 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:34.989659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:34.989746 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:34.989879 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:34.990477 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.991543 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.993679 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.994731 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.994935 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.995015 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:34.995422 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:44:34.995586 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 19:44:34.996953 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:34.988496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42655 (local address 127.0.198.194:42159)
0807 19:44:34.989312 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:34.989328 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:34.989512 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:34.989965 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:34.989968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:34.989990 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:34.990312 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.990320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.991711 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.991717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.993436 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:34.993452 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:34.994978 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:34.994985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:34.995177 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:34.995207 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:44:34.996169 (+ 962us) client_negotiation.cc:240] Negotiation successful
0807 19:44:34.996497 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":4,"spinlock_wait_cycles":8448,"thread_start_us":490,"threads_started":1}
I20260807 19:44:34.996840 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8def7eb90db74c7bafa0c99a662f6a36" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadd11c386ff4d86872dd35e4e69414b" is_pre_election: true
I20260807 19:44:34.997395 8322 raft_consensus.cc:2454] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 0.
I20260807 19:44:34.998090 8628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8def7eb90db74c7bafa0c99a662f6a36" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e163a25c62af4e6e858c26742b591eca" is_pre_election: true
I20260807 19:44:34.998248 8410 leader_election.cc:302] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cadd11c386ff4d86872dd35e4e69414b, dff9d30e63e343c4818e29779a7d06a3; no voters:
I20260807 19:44:34.998701 8628 raft_consensus.cc:2454] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 0.
I20260807 19:44:34.998808 8722 raft_consensus.cc:2781] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:34.999006 8722 raft_consensus.cc:482] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:34.999145 8722 raft_consensus.cc:3037] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:35.002033 8722 raft_consensus.cc:504] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:35.002698 8722 leader_election.cc:288] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 election: Requested vote from peers cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957), e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:35.003332 8628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8def7eb90db74c7bafa0c99a662f6a36" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e163a25c62af4e6e858c26742b591eca"
I20260807 19:44:35.003307 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8def7eb90db74c7bafa0c99a662f6a36" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadd11c386ff4d86872dd35e4e69414b"
I20260807 19:44:35.003585 8322 raft_consensus.cc:3037] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:35.003585 8628 raft_consensus.cc:3037] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:35.007190 8628 raft_consensus.cc:2454] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 1.
I20260807 19:44:35.007194 8322 raft_consensus.cc:2454] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 1.
I20260807 19:44:35.007948 8410 leader_election.cc:302] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cadd11c386ff4d86872dd35e4e69414b, dff9d30e63e343c4818e29779a7d06a3; no voters:
I20260807 19:44:35.008467 8722 raft_consensus.cc:2781] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:35.009119 8722 raft_consensus.cc:686] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 1 LEADER]: Becoming Leader. State: Replica: dff9d30e63e343c4818e29779a7d06a3, State: Running, Role: LEADER
I20260807 19:44:35.010001 8722 consensus_queue.cc:237] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
W20260807 19:44:35.012089 8531 tablet.cc:2369] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:35.016104 7341 catalog_manager.cc:5495] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "dff9d30e63e343c4818e29779a7d06a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:35.048051 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:35.042125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.0.198.194:45393)
0807 19:44:35.042381 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:35.042386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:35.042405 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:44:35.042506 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:35.042510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:35.042657 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:35.042849 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:35.043276 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.044204 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.045630 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.046049 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.046189 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:35.046779 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:35.047343 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:35.047581 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:35.047643 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 19:44:35.047736 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
W20260807 19:44:35.051239 8377 tablet.cc:2369] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:35.058051 8628 raft_consensus.cc:1261] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Refusing update from remote peer dff9d30e63e343c4818e29779a7d06a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:35.058051 8322 raft_consensus.cc:1261] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Refusing update from remote peer dff9d30e63e343c4818e29779a7d06a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:35.059314 8727 consensus_queue.cc:1035] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:35.059897 8722 consensus_queue.cc:1035] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:35.097637 7341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:35.098001 7341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:35.106112 7341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0c2ec4c9adf47618b38e0316eb1472f: Sending DeleteTablet for 3 replicas of tablet 8def7eb90db74c7bafa0c99a662f6a36
I20260807 19:44:35.107931 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:35 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:37442
I20260807 19:44:35.107944 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:35 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:39942
I20260807 19:44:35.109970 8608 tablet_service.cc:1514] Processing DeleteTablet for tablet 8def7eb90db74c7bafa0c99a662f6a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:35 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:34942
I20260807 19:44:35.118112 8742 tablet_replica.cc:330] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: stopping tablet replica
I20260807 19:44:35.120486 8742 raft_consensus.cc:2229] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:35.121533 8742 raft_consensus.cc:2258] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:35.123718 8743 tablet_replica.cc:330] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: stopping tablet replica
I20260807 19:44:35.125545 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7310
I20260807 19:44:35.126804 8744 tablet_replica.cc:330] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: stopping tablet replica
I20260807 19:44:35.126847 7374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:35.126973 8742 ts_tablet_manager.cc:1905] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:35.127295 8744 raft_consensus.cc:2229] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:35.127934 8744 raft_consensus.cc:2258] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:35.128481 8743 raft_consensus.cc:2229] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:35.129045 8743 raft_consensus.cc:2258] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:35.133505 8744 ts_tablet_manager.cc:1905] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:35.135496 8742 ts_tablet_manager.cc:1918] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:35.135718 8742 log.cc:1196] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/8def7eb90db74c7bafa0c99a662f6a36
I20260807 19:44:35.136373 8742 ts_tablet_manager.cc:1939] T 8def7eb90db74c7bafa0c99a662f6a36 P dff9d30e63e343c4818e29779a7d06a3: Deleting consensus metadata
I20260807 19:44:35.138458 7326 catalog_manager.cc:4841] TS dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393): tablet 8def7eb90db74c7bafa0c99a662f6a36 (table test-table [id=0eabd010ee764edeb86f8316f7dcaafc]) successfully deleted
I20260807 19:44:35.139865 8743 ts_tablet_manager.cc:1905] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:35.150102 8743 ts_tablet_manager.cc:1918] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:35.150315 8743 log.cc:1196] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/8def7eb90db74c7bafa0c99a662f6a36
I20260807 19:44:35.150588 8744 ts_tablet_manager.cc:1918] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:35.150825 8744 log.cc:1196] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/8def7eb90db74c7bafa0c99a662f6a36
I20260807 19:44:35.151505 8744 ts_tablet_manager.cc:1939] T 8def7eb90db74c7bafa0c99a662f6a36 P e163a25c62af4e6e858c26742b591eca: Deleting consensus metadata
I20260807 19:44:35.153975 8743 ts_tablet_manager.cc:1939] T 8def7eb90db74c7bafa0c99a662f6a36 P cadd11c386ff4d86872dd35e4e69414b: Deleting consensus metadata
I20260807 19:44:35.156749 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:35.118891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.0.198.253:34353)
0807 19:44:35.119104 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:35.119110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:35.124377 (+ 5267us) server_negotiation.cc:407] Connection header received
0807 19:44:35.135606 (+ 11229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:35.135616 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:35.135742 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:35.135833 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:35.139424 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.140978 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.143112 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.143799 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.143994 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:35.147891 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:35.147899 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:35.147904 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:35.147958 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:35.152306 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.153412 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.153418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.153422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.153587 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.154123 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.154129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.154132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.154546 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:35.154806 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:35.156108 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0807 19:44:35.156359 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:44:35.175529 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:35.124751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.0.198.252:41679)
0807 19:44:35.124987 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:35.124992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:35.157367 (+ 32375us) server_negotiation.cc:407] Connection header received
0807 19:44:35.157516 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:35.157522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:35.157613 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:35.157727 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:35.159677 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.161033 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.162999 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.163793 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.163959 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:35.167821 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:35.167828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:35.167834 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:35.167897 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:35.171845 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.172867 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.172873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.172877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.173057 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.173547 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.173552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.173556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.173960 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:35.174223 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:35.174629 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:44:35.174887 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 19:44:35.277077 7326 catalog_manager.cc:4841] TS cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957): tablet 8def7eb90db74c7bafa0c99a662f6a36 (table test-table [id=0eabd010ee764edeb86f8316f7dcaafc]) successfully deleted
I20260807 19:44:35.277514 7326 catalog_manager.cc:4841] TS e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655): tablet 8def7eb90db74c7bafa0c99a662f6a36 (table test-table [id=0eabd010ee764edeb86f8316f7dcaafc]) successfully deleted
I20260807 19:44:35.277985 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7310
I20260807 19:44:35.348939 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:35.324204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.0.198.252:41679)
0807 19:44:35.324450 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:35.324461 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:35.325703 (+ 1242us) server_negotiation.cc:407] Connection header received
0807 19:44:35.326040 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:35.326047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:35.326159 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:35.326256 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:35.329402 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.330929 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.333737 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.334645 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.334769 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:35.339704 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:35.339713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:35.339719 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:35.339772 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:35.344362 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.345413 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.345420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.345424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.345599 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.346839 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.346845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.346849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.347314 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:35.347633 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:35.348282 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 19:44:35.348553 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:44:35.352967 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:35.325317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35502 (local address 127.0.198.253:34353)
0807 19:44:35.325504 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:35.325510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:35.335162 (+ 9652us) server_negotiation.cc:407] Connection header received
0807 19:44:35.335413 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:35.335419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:35.335513 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:35.335639 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:35.338119 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.338965 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.341847 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:35.342429 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:35.343832 (+ 1403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:35.346085 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:35.346090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:35.346093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:35.346130 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:35.350039 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.350785 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.350791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.350795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.350962 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:35.351421 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:35.351428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:35.351437 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:35.351742 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:35.352038 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:35.352379 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 19:44:35.352623 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:44:36.084860 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.083785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:36807)
0807 19:44:36.084348 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.084515 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":297,"threads_started":1}
W20260807 19:44:36.085371 8679 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39287 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.094398 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.092364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:49917)
0807 19:44:36.093821 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.094064 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1273,"thread_start_us":1130,"threads_started":1}
W20260807 19:44:36.094988 8375 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39287 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.095489 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.092359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:48937)
0807 19:44:36.095048 (+ 2689us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.095258 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2566,"thread_start_us":2453,"threads_started":1}
W20260807 19:44:36.095849 8529 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39287 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.606040 8757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e0c2ec4c9adf47618b38e0316eb1472f)
I20260807 19:44:36.606406 8757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } }
I20260807 19:44:36.607910 8757 leader_election.cc:288] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287), 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253:34353)
I20260807 19:44:36.608454 8758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e0c2ec4c9adf47618b38e0316eb1472f)
I20260807 19:44:36.608670 8758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } }
I20260807 19:44:36.609531 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.608246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34574)
0807 19:44:36.608909 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.609151 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":289,"threads_started":1}
I20260807 19:44:36.609656 8758 leader_election.cc:288] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287), 071b89d70dfa4f91a348181543a4ba0c (127.0.198.252:41679)
W20260807 19:44:36.611143 8162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.611428 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.610086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34580)
0807 19:44:36.610753 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.611026 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":345,"threads_started":1}
W20260807 19:44:36.611677 7955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.617017 8766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.613000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34590)
0807 19:44:36.616432 (+ 3432us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.616691 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3254,"thread_start_us":3056,"threads_started":1}
W20260807 19:44:36.617972 7955 leader_election.cc:334] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287): Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.619302 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.618710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34604)
0807 19:44:36.618907 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.619108 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:44:36.619668 8162 leader_election.cc:334] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287): Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.621248 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.608973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34353 (local address 127.0.0.1:35506)
0807 19:44:36.609842 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.609867 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:36.610049 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:36.612531 (+ 2482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:36.612536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:36.612570 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:36.612891 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.612907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.615420 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.615431 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.618687 (+ 3256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.618703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.620339 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.620345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.620576 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.620616 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:44:36.620735 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:44:36.620906 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":1,"thread_start_us":1837,"threads_started":1}
I20260807 19:44:36.621487 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.609148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.0.198.253:34353)
0807 19:44:36.611439 (+ 2291us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:36.611445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:36.611469 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:44:36.612253 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:36.612265 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:36.612345 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:36.613038 (+ 693us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:36.613338 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.615048 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.618924 (+ 3876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.620160 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.620271 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.620360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:36.620912 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 19:44:36.621045 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2103,"thread_start_us":1890,"threads_started":1}
I20260807 19:44:36.622123 7981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071b89d70dfa4f91a348181543a4ba0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" is_pre_election: true
I20260807 19:44:36.622493 7981 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 071b89d70dfa4f91a348181543a4ba0c in term 1.
I20260807 19:44:36.623167 8162 leader_election.cc:302] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 071b89d70dfa4f91a348181543a4ba0c, 9058b5b7d85b43b4861d567af1f6f4ae; no voters: e0c2ec4c9adf47618b38e0316eb1472f
I20260807 19:44:36.623680 8757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:44:36.623852 8757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e0c2ec4c9adf47618b38e0316eb1472f)
I20260807 19:44:36.623957 8757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:44:36.624115 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.611697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:41679 (local address 127.0.0.1:54732)
0807 19:44:36.611958 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.611983 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:36.612091 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:36.617400 (+ 5309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:36.617405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:36.617438 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:36.617815 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.617831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.619688 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.619696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.621774 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.621790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.623395 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.623402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.623598 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.623635 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:44:36.623755 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 19:44:36.623922 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 19:44:36.624527 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.612631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.0.198.252:41679)
0807 19:44:36.616875 (+ 4244us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:36.616881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:36.616907 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:44:36.616982 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:36.616989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:36.617089 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:36.617372 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:36.618045 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.619468 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.621944 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.623148 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.623280 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.623477 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:36.623802 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 19:44:36.623973 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4086,"thread_start_us":4967,"threads_started":1}
I20260807 19:44:36.625375 8188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "071b89d70dfa4f91a348181543a4ba0c" is_pre_election: true
I20260807 19:44:36.627092 8757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } }
I20260807 19:44:36.627681 8188 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9058b5b7d85b43b4861d567af1f6f4ae in current term 2: Already voted for candidate 071b89d70dfa4f91a348181543a4ba0c in this term.
I20260807 19:44:36.628086 8757 leader_election.cc:288] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 election: Requested vote from peers e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287), 9058b5b7d85b43b4861d567af1f6f4ae (127.0.198.253:34353)
I20260807 19:44:36.628736 7954 leader_election.cc:302] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9058b5b7d85b43b4861d567af1f6f4ae; no voters: 071b89d70dfa4f91a348181543a4ba0c, e0c2ec4c9adf47618b38e0316eb1472f
I20260807 19:44:36.628883 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.628320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34610)
0807 19:44:36.628510 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.628698 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:44:36.629141 8758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:44:36.629894 7981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071b89d70dfa4f91a348181543a4ba0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9058b5b7d85b43b4861d567af1f6f4ae"
I20260807 19:44:36.630872 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.630381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34616)
0807 19:44:36.630553 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.630697 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:44:36.631033 8162 leader_election.cc:334] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287): Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
I20260807 19:44:36.632514 8758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:44:36.634991 7981 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 071b89d70dfa4f91a348181543a4ba0c in term 2.
I20260807 19:44:36.636052 8162 leader_election.cc:302] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 071b89d70dfa4f91a348181543a4ba0c, 9058b5b7d85b43b4861d567af1f6f4ae; no voters: e0c2ec4c9adf47618b38e0316eb1472f
I20260807 19:44:36.636479 8757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:44:36.637231 8757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [term 2 LEADER]: Becoming Leader. State: Replica: 071b89d70dfa4f91a348181543a4ba0c, State: Running, Role: LEADER
I20260807 19:44:36.637686 8757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } }
I20260807 19:44:36.639823 8767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071b89d70dfa4f91a348181543a4ba0c. Latest consensus state: current_term: 2 leader_uuid: "071b89d70dfa4f91a348181543a4ba0c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:36.640321 8767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:36.641248 8771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:36.646577 8771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:36.647620 8771 catalog_manager.cc:1259] Loaded cluster ID: a83fcd82a43c4b6a89bd1f0904f1191f
I20260807 19:44:36.647709 8771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:36.649065 8771 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:36.650032 8771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Loaded TSK: 0
I20260807 19:44:36.650938 8771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:44:36.660084 8177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:36.667992 8177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:36.673202 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.672640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34644)
0807 19:44:36.672897 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.673008 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:44:36.673214 7981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [term 2 FOLLOWER]: Refusing update from remote peer 071b89d70dfa4f91a348181543a4ba0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 19:44:36.674206 8757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 19:44:36.674750 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.674247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34654)
0807 19:44:36.674440 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.674562 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:44:36.674950 8162 consensus_peers.cc:487] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c -> Peer e0c2ec4c9adf47618b38e0316eb1472f (127.0.198.254:39287): Couldn't send request to peer e0c2ec4c9adf47618b38e0316eb1472f. Status: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:44:36.677878 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071b89d70dfa4f91a348181543a4ba0c. Latest consensus state: current_term: 2 leader_uuid: "071b89d70dfa4f91a348181543a4ba0c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:36.678267 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:36.680006 8757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "071b89d70dfa4f91a348181543a4ba0c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:36.680444 8757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:36.681504 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "071b89d70dfa4f91a348181543a4ba0c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0c2ec4c9adf47618b38e0316eb1472f" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39287 } } peers { permanent_uuid: "9058b5b7d85b43b4861d567af1f6f4ae" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34353 } attrs { promote: false } } peers { permanent_uuid: "071b89d70dfa4f91a348181543a4ba0c" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 41679 } attrs { promote: false } } }
I20260807 19:44:36.681892 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9058b5b7d85b43b4861d567af1f6f4ae [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:44:36.704074 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.692965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:32957 (local address 127.0.0.1:57280)
0807 19:44:36.693201 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.693233 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:36.693355 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:36.694560 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:36.694568 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:36.694605 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:36.694906 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.694921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.697105 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.697115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.699977 (+ 2862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.700004 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.702784 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.702795 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.703121 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.703189 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 19:44:36.703283 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 19:44:36.703539 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:44:36.705895 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.693164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.0.198.193:32957)
0807 19:44:36.694110 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:36.694120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:36.694144 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:44:36.694239 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:36.694248 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:36.694356 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:36.695087 (+ 731us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:36.695315 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.696857 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.700424 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.702476 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.704916 (+ 2440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.705105 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:36.705181 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 19:44:36.705405 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":691,"threads_started":1}
I20260807 19:44:36.707288 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 1a2f865db92748aaae664f0d02804046 (DEFAULT_TABLE table=test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:36.707854 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a2f865db92748aaae664f0d02804046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:36.713071 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.694220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.0.198.195:42655)
0807 19:44:36.706942 (+ 12722us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:36.706950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:36.706973 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:36.707049 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:36.707055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:36.707144 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:36.707257 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:36.707739 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.708801 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.711123 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.712000 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.712154 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.712280 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:36.712575 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:44:36.712734 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12593,"thread_start_us":12475,"threads_started":1}
I20260807 19:44:36.713095 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.693804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42655 (local address 127.0.0.1:55996)
0807 19:44:36.704469 (+ 10665us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.704498 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:36.704652 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:36.707316 (+ 2664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:36.707324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:36.707383 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:36.707565 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.707577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.708955 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.708961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.710944 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.710957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.712169 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.712175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.712403 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.712429 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:44:36.712545 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 19:44:36.712699 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10479,"mutex_wait_us":150,"thread_start_us":12475,"threads_started":1}
I20260807 19:44:36.713557 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.693433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:45393 (local address 127.0.0.1:42966)
0807 19:44:36.693596 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.693616 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:36.693735 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:36.706730 (+ 12995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:36.706737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:36.706772 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:36.707091 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.707103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.708713 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.708718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.711000 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:36.711016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.712761 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.712770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:36.712966 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.713007 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:44:36.713078 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 19:44:36.713248 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":63}
I20260807 19:44:36.714809 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.693966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.0.198.194:45393)
0807 19:44:36.706335 (+ 12369us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:36.706344 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:36.706369 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:44:36.706452 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:36.706457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:36.706544 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:36.707261 (+ 717us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:36.707459 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.708542 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.711232 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:36.712512 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:36.714135 (+ 1623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:36.714281 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:36.714336 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 19:44:36.714480 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12206,"thread_start_us":12503,"threads_started":1}
I20260807 19:44:36.715219 8782 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Bootstrap starting.
I20260807 19:44:36.715430 8608 tablet_service.cc:1467] Processing CreateTablet for tablet 1a2f865db92748aaae664f0d02804046 (DEFAULT_TABLE table=test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:36.715946 8608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a2f865db92748aaae664f0d02804046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:36.715859 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 1a2f865db92748aaae664f0d02804046 (DEFAULT_TABLE table=test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:36.716398 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a2f865db92748aaae664f0d02804046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:36.717689 8782 tablet_bootstrap.cc:654] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:36.720522 8782 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: No bootstrap required, opened a new log
I20260807 19:44:36.720685 8782 ts_tablet_manager.cc:1397] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:44:36.721532 8782 raft_consensus.cc:348] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.721882 8782 raft_consensus.cc:374] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:36.721998 8782 raft_consensus.cc:729] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cadd11c386ff4d86872dd35e4e69414b, State: Initialized, Role: FOLLOWER
I20260807 19:44:36.722306 8784 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Bootstrap starting.
I20260807 19:44:36.722298 8782 consensus_queue.cc:260] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.724715 8784 tablet_bootstrap.cc:654] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:36.725857 8786 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Bootstrap starting.
I20260807 19:44:36.726850 8782 ts_tablet_manager.cc:1428] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260807 19:44:36.727941 8786 tablet_bootstrap.cc:654] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:36.730440 8784 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: No bootstrap required, opened a new log
I20260807 19:44:36.730530 8786 tablet_bootstrap.cc:492] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: No bootstrap required, opened a new log
I20260807 19:44:36.730621 8784 ts_tablet_manager.cc:1397] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 19:44:36.730672 8786 ts_tablet_manager.cc:1397] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:44:36.731621 8784 raft_consensus.cc:348] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.731663 8786 raft_consensus.cc:348] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.731920 8786 raft_consensus.cc:374] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:36.731920 8784 raft_consensus.cc:374] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:36.731916 8374 heartbeater.cc:503] Master 127.0.198.252:41679 was elected leader, sending a full tablet report...
I20260807 19:44:36.732018 8786 raft_consensus.cc:729] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff9d30e63e343c4818e29779a7d06a3, State: Initialized, Role: FOLLOWER
I20260807 19:44:36.732077 8784 raft_consensus.cc:729] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e163a25c62af4e6e858c26742b591eca, State: Initialized, Role: FOLLOWER
I20260807 19:44:36.732311 8786 consensus_queue.cc:260] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.732362 8784 consensus_queue.cc:260] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:36.735839 8786 ts_tablet_manager.cc:1428] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 19:44:36.736164 8784 ts_tablet_manager.cc:1428] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260807 19:44:36.738070 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.728778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:37663)
0807 19:44:36.737558 (+ 8780us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.737755 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8599,"thread_start_us":8405,"threads_started":1}
I20260807 19:44:36.738698 8678 heartbeater.cc:503] Master 127.0.198.252:41679 was elected leader, sending a full tablet report...
I20260807 19:44:36.740492 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.739298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:46303)
0807 19:44:36.740214 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.740314 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":783}
I20260807 19:44:36.741858 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.736713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:41807)
0807 19:44:36.741425 (+ 4712us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.741589 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4556,"threads_started":1}
I20260807 19:44:36.743391 8528 heartbeater.cc:503] Master 127.0.198.252:41679 was elected leader, sending a full tablet report...
I20260807 19:44:36.743855 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.743425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:52369)
0807 19:44:36.743585 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.743692 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:44:36.744390 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.742139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:34419)
0807 19:44:36.743757 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.744056 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1489,"threads_started":1}
I20260807 19:44:36.745761 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:36.745269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:48161)
0807 19:44:36.745522 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:36.745611 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260807 19:44:37.055487 8792 raft_consensus.cc:482] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:37.055753 8792 raft_consensus.cc:504] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:37.056586 8792 leader_election.cc:288] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957), e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:37.057566 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a2f865db92748aaae664f0d02804046" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadd11c386ff4d86872dd35e4e69414b" is_pre_election: true
I20260807 19:44:37.057665 8628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a2f865db92748aaae664f0d02804046" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e163a25c62af4e6e858c26742b591eca" is_pre_election: true
I20260807 19:44:37.057868 8322 raft_consensus.cc:2454] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 0.
I20260807 19:44:37.058018 8628 raft_consensus.cc:2454] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 0.
I20260807 19:44:37.058440 8410 leader_election.cc:302] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cadd11c386ff4d86872dd35e4e69414b, dff9d30e63e343c4818e29779a7d06a3; no voters:
I20260807 19:44:37.058772 8792 raft_consensus.cc:2781] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:37.058909 8792 raft_consensus.cc:482] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:37.059022 8792 raft_consensus.cc:3037] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:37.061201 8792 raft_consensus.cc:504] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:37.061746 8792 leader_election.cc:288] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 election: Requested vote from peers cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957), e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655)
I20260807 19:44:37.062479 8628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a2f865db92748aaae664f0d02804046" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e163a25c62af4e6e858c26742b591eca"
I20260807 19:44:37.062484 8322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a2f865db92748aaae664f0d02804046" candidate_uuid: "dff9d30e63e343c4818e29779a7d06a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadd11c386ff4d86872dd35e4e69414b"
I20260807 19:44:37.062736 8628 raft_consensus.cc:3037] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:37.062736 8322 raft_consensus.cc:3037] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:37.066277 8322 raft_consensus.cc:2454] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 1.
I20260807 19:44:37.066295 8628 raft_consensus.cc:2454] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dff9d30e63e343c4818e29779a7d06a3 in term 1.
I20260807 19:44:37.066907 8410 leader_election.cc:302] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dff9d30e63e343c4818e29779a7d06a3, e163a25c62af4e6e858c26742b591eca; no voters:
I20260807 19:44:37.067328 8792 raft_consensus.cc:2781] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:37.067898 8792 raft_consensus.cc:686] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 1 LEADER]: Becoming Leader. State: Replica: dff9d30e63e343c4818e29779a7d06a3, State: Running, Role: LEADER
I20260807 19:44:37.068248 8792 consensus_queue.cc:237] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } }
I20260807 19:44:37.069352 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.068751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:51215)
0807 19:44:37.068998 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.069110 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:44:37.071702 8176 catalog_manager.cc:5495] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "dff9d30e63e343c4818e29779a7d06a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dff9d30e63e343c4818e29779a7d06a3" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 45393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:37.172214 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.166816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.0.198.194:45393)
0807 19:44:37.167031 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:37.167037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:37.167263 (+ 226us) server_negotiation.cc:407] Connection header received
0807 19:44:37.167433 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:37.167437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:37.167512 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:37.167616 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:37.168112 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.168908 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.170329 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.170829 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.170947 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:37.171531 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:37.171668 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:37.171756 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:37.171915 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:44:37.171991 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:44:37.175928 8628 raft_consensus.cc:1261] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Refusing update from remote peer dff9d30e63e343c4818e29779a7d06a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:37.175980 8322 raft_consensus.cc:1261] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Refusing update from remote peer dff9d30e63e343c4818e29779a7d06a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:37.176791 8796 consensus_queue.cc:1035] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e163a25c62af4e6e858c26742b591eca" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:37.177214 8792 consensus_queue.cc:1035] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cadd11c386ff4d86872dd35e4e69414b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:37.180132 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.179607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:50319)
0807 19:44:37.179802 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.179932 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:44:37.181348 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.180632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:40325)
0807 19:44:37.180899 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.181030 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:44:37.185407 8800 mvcc.cc:205] Tried to move back new op lower bound from 7315996168905486336 to 7315996168474673152. Current Snapshot: MvccSnapshot[applied={T|T < 7315996168905486336}]
I20260807 19:44:37.188124 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.187266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.195:54533)
0807 19:44:37.187805 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.187919 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401}
I20260807 19:44:37.188349 8802 mvcc.cc:205] Tried to move back new op lower bound from 7315996168905486336 to 7315996168474673152. Current Snapshot: MvccSnapshot[applied={T|T < 7315996168905486336}]
I20260807 19:44:37.190676 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.188579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:35981)
0807 19:44:37.190388 (+ 1809us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.190508 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1667}
I20260807 19:44:37.192812 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.191807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.193:49481)
0807 19:44:37.192138 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.192253 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
I20260807 19:44:37.195032 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.194479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.198.194:36879)
0807 19:44:37.194675 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.194813 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:44:37.197494 8176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:37.197764 8176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:37.203900 8176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 071b89d70dfa4f91a348181543a4ba0c: Sending DeleteTablet for 3 replicas of tablet 1a2f865db92748aaae664f0d02804046
I20260807 19:44:37.205118 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a2f865db92748aaae664f0d02804046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:37 UTC) from {username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:57280
I20260807 19:44:37.205302 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a2f865db92748aaae664f0d02804046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:37 UTC) from {username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:42966
I20260807 19:44:37.205343 8608 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a2f865db92748aaae664f0d02804046 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:37 UTC) from {username='oryx', principal='oryx/127.0.198.252@KRBTEST.COM'} at 127.0.0.1:55996
I20260807 19:44:37.207010 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 8148
I20260807 19:44:37.208578 8808 tablet_replica.cc:330] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: stopping tablet replica
I20260807 19:44:37.208884 8808 raft_consensus.cc:2229] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:37.209245 8808 raft_consensus.cc:2258] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:37.210031 8810 tablet_replica.cc:330] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: stopping tablet replica
I20260807 19:44:37.210330 8810 raft_consensus.cc:2229] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:37.210927 8810 raft_consensus.cc:2258] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:37.212697 8808 ts_tablet_manager.cc:1905] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:37.216396 8810 ts_tablet_manager.cc:1905] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:37.224442 8809 tablet_replica.cc:330] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: stopping tablet replica
I20260807 19:44:37.224831 8808 ts_tablet_manager.cc:1918] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:37.224957 8809 raft_consensus.cc:2229] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:37.225082 8808 log.cc:1196] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/1a2f865db92748aaae664f0d02804046
I20260807 19:44:37.225387 8809 raft_consensus.cc:2258] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:37.225837 8808 ts_tablet_manager.cc:1939] T 1a2f865db92748aaae664f0d02804046 P cadd11c386ff4d86872dd35e4e69414b: Deleting consensus metadata
I20260807 19:44:37.226888 8809 ts_tablet_manager.cc:1905] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:37.227480 8810 ts_tablet_manager.cc:1918] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:37.227732 8810 log.cc:1196] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/1a2f865db92748aaae664f0d02804046
I20260807 19:44:37.228312 8810 ts_tablet_manager.cc:1939] T 1a2f865db92748aaae664f0d02804046 P dff9d30e63e343c4818e29779a7d06a3: Deleting consensus metadata
I20260807 19:44:37.230083 8161 catalog_manager.cc:4841] TS dff9d30e63e343c4818e29779a7d06a3 (127.0.198.194:45393): tablet 1a2f865db92748aaae664f0d02804046 (table test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]) successfully deleted
I20260807 19:44:37.232577 8161 catalog_manager.cc:4841] TS cadd11c386ff4d86872dd35e4e69414b (127.0.198.193:32957): tablet 1a2f865db92748aaae664f0d02804046 (table test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]) successfully deleted
I20260807 19:44:37.237161 8809 ts_tablet_manager.cc:1918] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:37.237365 8809 log.cc:1196] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/1a2f865db92748aaae664f0d02804046
I20260807 19:44:37.237958 8809 ts_tablet_manager.cc:1939] T 1a2f865db92748aaae664f0d02804046 P e163a25c62af4e6e858c26742b591eca: Deleting consensus metadata
I20260807 19:44:37.238004 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.227070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34670)
0807 19:44:37.237556 (+ 10486us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.237759 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10317,"threads_started":1}
I20260807 19:44:37.239564 8161 catalog_manager.cc:4841] TS e163a25c62af4e6e858c26742b591eca (127.0.198.195:42655): tablet 1a2f865db92748aaae664f0d02804046 (table test-table [id=95c4f3dfec2d42a1a3a15150bcfe0b43]) successfully deleted
I20260807 19:44:37.241350 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.240854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34678)
0807 19:44:37.241044 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.241142 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:44:37.242256 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.208124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.0.198.193:32957)
0807 19:44:37.224018 (+ 15894us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:37.224029 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:37.224063 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:44:37.224145 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:37.224150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:37.224273 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:37.224443 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:37.226613 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.227781 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.230265 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.230894 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.231062 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:37.234556 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:37.234576 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:37.234594 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:37.234635 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:37.238422 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.239282 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.239294 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.239297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.239524 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.240039 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.240046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.240049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.240426 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:37.240781 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:37.241770 (+ 989us) server_negotiation.cc:299] Negotiation successful
0807 19:44:37.241982 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15590,"thread_start_us":15390,"threads_started":1}
I20260807 19:44:37.243140 8367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:37.344815 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 8148
I20260807 19:44:37.367758 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 8392
I20260807 19:44:37.382355 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.368610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.0.198.194:45393)
0807 19:44:37.368808 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:37.368813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:37.368834 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:44:37.368920 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:37.368924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:37.369007 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:37.369099 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:37.370512 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.371265 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.372730 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.373146 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.373267 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:37.376457 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:37.376474 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:37.376491 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:37.376529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:37.379692 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.380443 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.380451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.380453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.380590 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.380973 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.380976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.380978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.381362 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:37.381589 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:37.381906 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:44:37.382116 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:44:37.383122 8523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:37.515870 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 8392
I20260807 19:44:37.544242 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 8541
I20260807 19:44:37.560879 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.545252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.0.198.195:42655)
0807 19:44:37.545892 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:37.545897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:37.545914 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:44:37.545971 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:37.545975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:37.546050 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:37.546185 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:37.548005 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.548800 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.550397 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:37.550897 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:37.551061 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:37.554085 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:37.554106 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:37.554127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:37.554178 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:37.557472 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.558389 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.558398 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.558401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.558553 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:37.558986 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:37.558992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:37.558995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:37.559478 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:37.559868 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:37.560184 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 19:44:37.560445 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":305,"threads_started":1}
I20260807 19:44:37.561676 8672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:37.648825 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.648269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34684)
0807 19:44:37.648485 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.648597 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:44:37.650017 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:37.649605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39287 (local address 127.0.0.1:34688)
0807 19:44:37.649780 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:37.649871 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:44:37.660053 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 8541
I20260807 19:44:37.681627 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7826
I20260807 19:44:37.682814 8002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:37.806465 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7826
I20260807 19:44:37.836520 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 7933
I20260807 19:44:37.837800 8211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:37.985922 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 7933
2026-08-07T19:44:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10804 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:38 dist-test-slave-btw8 krb5kdc[8825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:38 dist-test-slave-btw8 krb5kdc[8825](info): set up 2 sockets
Aug 07 19:44:38 dist-test-slave-btw8 krb5kdc[8825](info): commencing operation
krb5kdc: starting...
W20260807 19:44:39.198881 795 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:39 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab.
Entry for principal oryx/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab.
I20260807 19:44:39.524322 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:45513
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab
--principal=oryx/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:45513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:39.910943 8838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:39.911425 8838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:39.911614 8838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:39.920447 8838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:39.920640 8838 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:44:39.920728 8838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:39.920822 8838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:39.920921 8838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:39.921013 8838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:39.933750 8838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:45513
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.254.keytab
--principal=oryx/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:45513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:39.935277 8838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:39.936940 8838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:39.951136 8843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:39.951092 8847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:39.951159 8845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:39.951998 8838 server_base.cc:1034] running on GCE node
I20260807 19:44:39.952906 8838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:39.954607 8838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:39.955899 8838 hybrid_clock.cc:648] HybridClock initialized: now 1786131879955819 us; error 55 us; skew 500 ppm
I20260807 19:44:39.956332 8838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:39 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131879, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:39.963969 8838 init.cc:377] Logged in from keytab as oryx/127.0.198.254@KRBTEST.COM (short username oryx)
I20260807 19:44:39.965993 8838 webserver.cc:460] Webserver started at http://127.0.198.254:34635/ using document root <none> and password file <none>
I20260807 19:44:39.966840 8838 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:39.966955 8838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:39.967309 8838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:39.971099 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "7e771646645f4a4686696e1c1f7d3231"
format_stamp: "Formatted at 2026-08-07 19:44:39 on dist-test-slave-btw8"
server_key: "00397cbe278d5ec42e0672bce5327d38"
server_key_iv: "643cb48d06116d93c05b0526ae208fff"
server_key_version: "encryptionkey@0"
I20260807 19:44:39.972112 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "7e771646645f4a4686696e1c1f7d3231"
format_stamp: "Formatted at 2026-08-07 19:44:39 on dist-test-slave-btw8"
server_key: "00397cbe278d5ec42e0672bce5327d38"
server_key_iv: "643cb48d06116d93c05b0526ae208fff"
server_key_version: "encryptionkey@0"
I20260807 19:44:39.977977 8838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:44:39.982023 8854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:39.983775 8838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:44:39.983940 8838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "7e771646645f4a4686696e1c1f7d3231"
format_stamp: "Formatted at 2026-08-07 19:44:39 on dist-test-slave-btw8"
server_key: "00397cbe278d5ec42e0672bce5327d38"
server_key_iv: "643cb48d06116d93c05b0526ae208fff"
server_key_version: "encryptionkey@0"
I20260807 19:44:39.984159 8838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:40.012682 8838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:40.017964 8838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:40.018235 8838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:40.043929 8838 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:45513
I20260807 19:44:40.044045 8916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:45513 every 8 connection(s)
I20260807 19:44:40.045451 8838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:40.049053 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 8838
I20260807 19:44:40.049319 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:40.050271 795 external_mini_cluster.cc:1428] Setting key 2a1356940da774ee042c5896cf185712
I20260807 19:44:40.050817 8919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:40.069410 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Bootstrap starting.
Aug 07 19:44:40 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:40.073625 8919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:40.074949 8919 log.cc:824] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:40.078593 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: No bootstrap required, opened a new log
I20260807 19:44:40.079708 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:40.052935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.0.198.254:45513)
0807 19:44:40.053950 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:40.053964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:40.054019 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:44:40.056813 (+ 2794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:40.056845 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:40.057383 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:40.057811 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:40.060507 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.062004 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.064229 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.065110 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.065308 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:40.071056 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:40.071066 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:40.071088 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:40.071150 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:40.074769 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.075808 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.075820 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.075827 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.076029 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.076515 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.076520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.076522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.077107 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:40.077463 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:40.077950 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 19:44:40.078440 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":506,"threads_started":1}
I20260807 19:44:40.088930 8919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } }
I20260807 19:44:40.089504 8919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:40.089650 8919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e771646645f4a4686696e1c1f7d3231, State: Initialized, Role: FOLLOWER
I20260807 19:44:40.090523 8919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } }
I20260807 19:44:40.090862 8919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:40.091053 8919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:40.091245 8919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:40.094295 8919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } }
I20260807 19:44:40.094940 8919 leader_election.cc:302] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e771646645f4a4686696e1c1f7d3231; no voters:
I20260807 19:44:40.095674 8919 leader_election.cc:288] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:40.095767 8924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:40.096657 8924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [term 1 LEADER]: Becoming Leader. State: Replica: 7e771646645f4a4686696e1c1f7d3231, State: Running, Role: LEADER
I20260807 19:44:40.097414 8924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } }
I20260807 19:44:40.098268 8919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:40.100612 8925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e771646645f4a4686696e1c1f7d3231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } } }
I20260807 19:44:40.101045 8925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:40.100813 8926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e771646645f4a4686696e1c1f7d3231. Latest consensus state: current_term: 1 leader_uuid: "7e771646645f4a4686696e1c1f7d3231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e771646645f4a4686696e1c1f7d3231" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 45513 } } }
I20260807 19:44:40.101234 8926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:40.103513 8931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:40.109338 8931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:40.120355 8931 catalog_manager.cc:1347] Generated new cluster ID: ae1f52c4002a4140b00bcb1de374f0bb
I20260807 19:44:40.120504 8931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:40.137943 8931 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:40.140012 8931 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:40.157636 8931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Generated new TSK 0
I20260807 19:44:40.158746 8931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
I20260807 19:44:40.244786 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:45513
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:40.617873 8948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:40.618207 8948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:40.618315 8948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:40.626142 8948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:40.626305 8948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:40.626401 8948 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:44:40.626606 8948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:40.637120 8948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:45513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:40.638477 8948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:40.639873 8948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:40.654366 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:40.654460 8954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:40.654482 8956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:40.655512 8948 server_base.cc:1034] running on GCE node
I20260807 19:44:40.656191 8948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:40.657445 8948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:40.658733 8948 hybrid_clock.cc:648] HybridClock initialized: now 1786131880658636 us; error 73 us; skew 500 ppm
I20260807 19:44:40.659101 8948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:40 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131880, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:40.662878 8948 init.cc:377] Logged in from keytab as oryx/127.0.198.193@KRBTEST.COM (short username oryx)
I20260807 19:44:40.664572 8948 webserver.cc:460] Webserver started at http://127.0.198.193:41231/ using document root <none> and password file <none>
I20260807 19:44:40.665407 8948 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:40.665561 8948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:40.665939 8948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:40.669337 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "8059002e5dc24dddbf6832eef87ace67"
format_stamp: "Formatted at 2026-08-07 19:44:40 on dist-test-slave-btw8"
server_key: "65624efcb9536dbdc7d82ffe40edef59"
server_key_iv: "998030ca60b87c865dcf64fa1058faad"
server_key_version: "encryptionkey@0"
I20260807 19:44:40.670279 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "8059002e5dc24dddbf6832eef87ace67"
format_stamp: "Formatted at 2026-08-07 19:44:40 on dist-test-slave-btw8"
server_key: "65624efcb9536dbdc7d82ffe40edef59"
server_key_iv: "998030ca60b87c865dcf64fa1058faad"
server_key_version: "encryptionkey@0"
I20260807 19:44:40.676261 8948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:44:40.680471 8964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:40.682271 8948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:44:40.682541 8948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "8059002e5dc24dddbf6832eef87ace67"
format_stamp: "Formatted at 2026-08-07 19:44:40 on dist-test-slave-btw8"
server_key: "65624efcb9536dbdc7d82ffe40edef59"
server_key_iv: "998030ca60b87c865dcf64fa1058faad"
server_key_version: "encryptionkey@0"
I20260807 19:44:40.682843 8948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:40.715389 8948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:40.720909 8948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:40.721257 8948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:40.724856 8948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:40.724993 8948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:40.725171 8948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:40.725255 8948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:44:40 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131880, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:40.753285 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:40.725691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:58829 (local address 127.0.198.254:45513)
0807 19:44:40.729074 (+ 3383us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:40.729079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:40.729102 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:40.732076 (+ 2974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:40.732086 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:40.732212 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:40.732354 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:40.735332 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.736515 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.739073 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.739941 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.740126 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:40.744782 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:40.744792 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:40.744800 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:40.744866 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:40.748723 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.749944 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.749952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.749960 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.750094 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.750782 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.750790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.750795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.751278 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:40.751478 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:40.752633 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0807 19:44:40.752891 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3251,"thread_start_us":3114,"threads_started":1}
I20260807 19:44:40.755087 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:40.726686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.193:58829)
0807 19:44:40.727905 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:40.727974 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:40.731584 (+ 3610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:40.732804 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:40.732820 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:40.733831 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:40.735047 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:40.735096 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.736703 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.736709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:40.738814 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:40.738838 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.740268 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.740275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:40.740508 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:40.741328 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:40.741354 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:40.744414 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:40.748909 (+ 4495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:40.748920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:40.748945 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:40.749782 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:40.750239 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:40.750247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:40.750256 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:40.750634 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:40.751480 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:40.751492 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:40.752053 (+ 561us) client_negotiation.cc:769] Sending connection context
0807 19:44:40.752529 (+ 476us) client_negotiation.cc:240] Negotiation successful
0807 19:44:40.753196 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":517,"threads_started":1}
I20260807 19:44:40.775413 8948 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:44639
I20260807 19:44:40.775439 9091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:44639 every 8 connection(s)
I20260807 19:44:40.777216 8948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:40.779765 9092 heartbeater.cc:347] Connected to a master server at 127.0.198.254:45513
I20260807 19:44:40.780092 9092 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:40.780139 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 8948
I20260807 19:44:40.780257 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:40.762713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:55669 (local address 127.0.198.254:45513)
0807 19:44:40.762906 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:40.762911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:40.763709 (+ 798us) server_negotiation.cc:407] Connection header received
0807 19:44:40.763966 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:40.763974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:40.764096 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:40.764244 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:40.765759 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.767095 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.769404 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.769866 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.770016 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:40.772798 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:40.772808 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:40.772816 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:40.772877 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:40.776812 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.777684 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.777692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.777698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.777889 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:40.778395 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:40.778402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:40.778407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:40.778842 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:40.779025 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:40.779587 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 19:44:40.779872 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:44:40.780378 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:40.780396 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:40.762719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.193:55669)
0807 19:44:40.763552 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:40.763578 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:40.763779 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:40.764290 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:40.764294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:40.764945 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:40.765546 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:40.765566 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.767299 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.767310 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:40.769204 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:40.769223 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:40.770093 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:40.770101 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:40.770266 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:40.771114 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:40.771133 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:40.772489 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:40.776987 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:40.776992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:40.776995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:40.777524 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:40.778027 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:40.778032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:40.778035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:40.778213 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:40.778999 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:40.779006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:40.779221 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 19:44:40.779431 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 19:44:40.779843 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":399,"threads_started":1}
I20260807 19:44:40.780918 9092 heartbeater.cc:511] Master 127.0.198.254:45513 requested a full tablet report, sending...
I20260807 19:44:40.781080 795 external_mini_cluster.cc:1428] Setting key 4f4864d693794797edf205d46ac7c573
I20260807 19:44:40.784320 8869 ts_manager.cc:194] Registered new tserver with Master: 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639)
I20260807 19:44:40.786692 8869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:58829
WARNING: no policy specified for oryx/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
I20260807 19:44:40.847486 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:45513
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:41.210637 9096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:41.210961 9096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:41.211061 9096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:41.218763 9096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:41.218905 9096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:41.218994 9096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:44:41.219089 9096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:41.229422 9096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:45513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:41.230783 9096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:41.232137 9096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:41.246481 9101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:41.246563 9104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:41.246595 9102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:41.248102 9096 server_base.cc:1034] running on GCE node
I20260807 19:44:41.249064 9096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:41.250168 9096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:41.251534 9096 hybrid_clock.cc:648] HybridClock initialized: now 1786131881251405 us; error 62 us; skew 500 ppm
I20260807 19:44:41.251931 9096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:41 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131881, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:41.255838 9096 init.cc:377] Logged in from keytab as oryx/127.0.198.194@KRBTEST.COM (short username oryx)
I20260807 19:44:41.257805 9096 webserver.cc:460] Webserver started at http://127.0.198.194:42345/ using document root <none> and password file <none>
I20260807 19:44:41.258772 9096 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:41.258939 9096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:41.259305 9096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:41.263115 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "38a92ec4230743f5b8061e8e7525e409"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "99b0b654cdaa4f24556ba013f4e80bc4"
server_key_iv: "58cfd5ef2b0469c01dbeb4e916c70472"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.264171 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "38a92ec4230743f5b8061e8e7525e409"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "99b0b654cdaa4f24556ba013f4e80bc4"
server_key_iv: "58cfd5ef2b0469c01dbeb4e916c70472"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.270843 9096 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 19:44:41.275012 9111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:41.276643 9096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:41.276862 9096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "38a92ec4230743f5b8061e8e7525e409"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "99b0b654cdaa4f24556ba013f4e80bc4"
server_key_iv: "58cfd5ef2b0469c01dbeb4e916c70472"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.277138 9096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:41.290601 9096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:41.295409 9096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:41.295732 9096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:41.298728 9096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:41.298851 9096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:41.299019 9096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:41.299145 9096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 19:44:41 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131881, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:41.329578 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.299499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:51091 (local address 127.0.198.254:45513)
0807 19:44:41.302572 (+ 3073us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:41.302578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:41.302607 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:44:41.305344 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:41.305351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:41.305459 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:41.305567 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:41.308665 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.309903 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.313146 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.313864 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.314046 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.320417 (+ 6371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:41.320424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:41.320427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:41.320475 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:41.324718 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.326300 (+ 1582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.326307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.326310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.326463 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.327140 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.327144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.327146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.327520 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:41.327710 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:41.328908 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0807 19:44:41.329218 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2952,"thread_start_us":2834,"threads_started":1}
I20260807 19:44:41.331694 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.300425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.194:51091)
0807 19:44:41.301506 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:41.301565 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:41.304871 (+ 3306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:41.305948 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:41.305965 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:41.307038 (+ 1073us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:41.308291 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.308336 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.310142 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.310150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.312896 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.312925 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.314212 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.314223 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.314686 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.315863 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:41.315915 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:41.320048 (+ 4133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:41.325022 (+ 4974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.325039 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.325067 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.326089 (+ 1022us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.326596 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.326604 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.326611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.326965 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.327737 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:41.327747 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:41.328330 (+ 583us) client_negotiation.cc:769] Sending connection context
0807 19:44:41.329847 (+ 1517us) client_negotiation.cc:240] Negotiation successful
0807 19:44:41.330380 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":406,"threads_started":1}
I20260807 19:44:41.357517 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.338907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:52459 (local address 127.0.198.254:45513)
0807 19:44:41.339170 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:41.339181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:41.339790 (+ 609us) server_negotiation.cc:407] Connection header received
0807 19:44:41.339968 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:41.339977 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:41.340119 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:41.340274 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:41.341419 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.342782 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.345272 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.346077 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.346282 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.349050 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:41.349059 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:41.349066 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:41.349122 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:41.353275 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.354495 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.354502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.354505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.354710 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.355330 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.355338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.355342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.355849 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:41.356089 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:41.356797 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 19:44:41.357043 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:44:41.357790 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.338871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.194:52459)
0807 19:44:41.339652 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:41.339675 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:41.339831 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:41.340337 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:41.340340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:41.340916 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:41.341237 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.341247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.342981 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.342987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.345033 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.345075 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.346329 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.346342 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.346566 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.347476 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:41.347511 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:41.348766 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:41.353576 (+ 4810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.353589 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.353593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.354295 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.354893 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.354902 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.354908 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.355142 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.356012 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:41.356021 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:41.356376 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 19:44:41.356589 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 19:44:41.356988 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":419,"threads_started":1}
I20260807 19:44:41.360270 9096 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:44949
I20260807 19:44:41.360296 9239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:44949 every 8 connection(s)
I20260807 19:44:41.362037 9096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:41.364961 9240 heartbeater.cc:347] Connected to a master server at 127.0.198.254:45513
I20260807 19:44:41.365254 9240 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:41.366007 9240 heartbeater.cc:511] Master 127.0.198.254:45513 requested a full tablet report, sending...
I20260807 19:44:41.367792 8869 ts_manager.cc:194] Registered new tserver with Master: 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949)
I20260807 19:44:41.368680 8869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:51091
I20260807 19:44:41.371758 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9096
I20260807 19:44:41.371963 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:41.372465 795 external_mini_cluster.cc:1428] Setting key b39a9c7ee780650e7f418a39dec221ee
WARNING: no policy specified for oryx/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
I20260807 19:44:41.439888 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:45513
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:41.790118 9092 heartbeater.cc:503] Master 127.0.198.254:45513 was elected leader, sending a full tablet report...
W20260807 19:44:41.803769 9244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:41.804134 9244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:41.804240 9244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:41.812022 9244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:41.812213 9244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:41.812361 9244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:44:41.812500 9244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:41.824532 9244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.198.254:45513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:41.826009 9244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:41.827559 9244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:41.842334 9253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:41.842293 9251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:41.842358 9250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:41.843473 9244 server_base.cc:1034] running on GCE node
I20260807 19:44:41.844311 9244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:41.845424 9244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:41.846720 9244 hybrid_clock.cc:648] HybridClock initialized: now 1786131881846660 us; error 42 us; skew 500 ppm
I20260807 19:44:41.847131 9244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:41 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131881, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:41.850962 9244 init.cc:377] Logged in from keytab as oryx/127.0.198.195@KRBTEST.COM (short username oryx)
I20260807 19:44:41.852782 9244 webserver.cc:460] Webserver started at http://127.0.198.195:37707/ using document root <none> and password file <none>
I20260807 19:44:41.853567 9244 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:41.853668 9244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:41.854156 9244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:41.857945 9244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "ecfe78da89dd46d9a32b072d34dbf605"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "358cb6048b4e9e668690ff4fdc3cd23a"
server_key_iv: "df6edc64e3f2e6ad7b6dd9b74799e4ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.859054 9244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "ecfe78da89dd46d9a32b072d34dbf605"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "358cb6048b4e9e668690ff4fdc3cd23a"
server_key_iv: "df6edc64e3f2e6ad7b6dd9b74799e4ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.865648 9244 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:44:41.870254 9260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:41.872162 9244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:41.872370 9244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "ecfe78da89dd46d9a32b072d34dbf605"
format_stamp: "Formatted at 2026-08-07 19:44:41 on dist-test-slave-btw8"
server_key: "358cb6048b4e9e668690ff4fdc3cd23a"
server_key_iv: "df6edc64e3f2e6ad7b6dd9b74799e4ae"
server_key_version: "encryptionkey@0"
I20260807 19:44:41.872613 9244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:41.901595 9244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:41.907624 9244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:41.907948 9244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:41.912118 9244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:41.912261 9244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:41.912406 9244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:41.912489 9244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:44:41 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131881, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:41.943516 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.913296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:51599 (local address 127.0.198.254:45513)
0807 19:44:41.913831 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:41.913840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:41.915448 (+ 1608us) server_negotiation.cc:407] Connection header received
0807 19:44:41.918406 (+ 2958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:41.918414 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:41.918581 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:41.918697 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:41.922251 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.924253 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.926837 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.927765 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.927940 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.933692 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:41.933702 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:41.933710 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:41.933773 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:41.938344 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.939823 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.939831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.939840 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.940073 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.940929 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.940937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.940941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.941305 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:41.941506 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:41.942861 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0807 19:44:41.943128 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
I20260807 19:44:41.946009 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.914086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.195:51599)
0807 19:44:41.915168 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:41.915242 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:41.917765 (+ 2523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:41.919416 (+ 1651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:41.919432 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:41.920567 (+ 1135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:41.921764 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.921840 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.925053 (+ 3213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.925060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.926579 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.926611 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.928101 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.928109 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.928555 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.929623 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:41.929668 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:41.933309 (+ 3641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:41.938619 (+ 5310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.938636 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.938666 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.939564 (+ 898us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.940276 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.940285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.940294 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.940729 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.941510 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:41.941525 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:41.942240 (+ 715us) client_negotiation.cc:769] Sending connection context
0807 19:44:41.942701 (+ 461us) client_negotiation.cc:240] Negotiation successful
0807 19:44:41.943584 (+ 883us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":464,"threads_started":1}
I20260807 19:44:41.970738 9244 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:35371
I20260807 19:44:41.970738 9388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:35371 every 8 connection(s)
I20260807 19:44:41.972652 9244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:41.972713 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.954260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.198.195:34579)
0807 19:44:41.955271 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:41.955296 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:41.955638 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:41.956123 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:41.956131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:41.957295 (+ 1164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:41.957800 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.957823 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.959501 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.959512 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.961969 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:41.961998 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.962933 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.962948 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:41.963187 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.964281 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:41.964301 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:41.965844 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:41.969467 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.969473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.969481 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.970255 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.970697 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:41.970702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:41.970706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:41.970867 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:41.971713 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:41.971717 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:41.971899 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 19:44:41.972036 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:44:41.972369 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":438,"threads_started":1}
I20260807 19:44:41.972859 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:41.954168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:34579 (local address 127.0.198.254:45513)
0807 19:44:41.954411 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:41.954421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:41.955477 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 19:44:41.955839 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:41.955853 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:41.955935 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:41.956038 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:41.958114 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.959244 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.962211 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:41.962663 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:41.962781 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:41.966123 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:41.966129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:41.966133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:41.966170 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:41.969266 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.970400 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.970408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.970417 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.970561 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:41.971062 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:41.971069 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:41.971072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:41.971542 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:41.971735 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:41.972203 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 19:44:41.972474 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:44:41.974987 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9244
I20260807 19:44:41.975176 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:41.975733 795 external_mini_cluster.cc:1428] Setting key 1fa69c2ea164b44cacbad565f616f810
I20260807 19:44:41.977618 9389 heartbeater.cc:347] Connected to a master server at 127.0.198.254:45513
I20260807 19:44:41.977916 9389 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:41.978639 9389 heartbeater.cc:511] Master 127.0.198.254:45513 requested a full tablet report, sending...
I20260807 19:44:41.980165 8869 ts_manager.cc:194] Registered new tserver with Master: ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:41.981019 8869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:51599
I20260807 19:44:41.993075 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:44:42 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131882, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:44:42 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:42 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:42.371429 9240 heartbeater.cc:503] Master 127.0.198.254:45513 was elected leader, sending a full tablet report...
I20260807 19:44:42.984371 9389 heartbeater.cc:503] Master 127.0.198.254:45513 was elected leader, sending a full tablet report...
W20260807 19:44:43.269609 9397 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.208s user 0.004s sys 0.018s
W20260807 19:44:43.270871 9270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:42.055092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.0.198.254:45513)
0807 19:44:42.055434 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:42.055442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:42.055685 (+ 243us) server_negotiation.cc:407] Connection header received
0807 19:44:42.055840 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:42.055844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:42.055935 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:42.056036 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:42.057671 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:42.058619 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:42.060246 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:42.060724 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:42.060840 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.270624 (+1209784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41708: BlockingRecv error: recv got EOF from 127.0.0.1:41708 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 19:44:43 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131882, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:43.291132 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.275342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.0.198.254:45513)
0807 19:44:43.275623 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.275630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.275874 (+ 244us) server_negotiation.cc:407] Connection header received
0807 19:44:43.276102 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.276108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.276193 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.276333 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:43.277844 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.278610 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.280088 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.280543 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.280696 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.285004 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:43.285010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:43.285013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:43.285058 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:43.288318 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:43.289310 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:43.289315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:43.289317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:43.289458 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:43.289840 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:43.289843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:43.289846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:43.290129 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:43.290369 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.290684 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.290894 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:44:43.295598 8869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:43.302961 8869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:43.330096 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.317444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:35371 (local address 127.0.0.1:60352)
0807 19:44:43.318358 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:43.318409 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:43.318584 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:43.321201 (+ 2617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:43.321209 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:43.321245 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:43.321818 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.321839 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.324136 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.324142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.326825 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.326840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.328986 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.328993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.329253 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.329338 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 19:44:43.329457 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:44:43.329677 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":810,"threads_started":1}
I20260807 19:44:43.330413 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.318320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.0.198.193:44639)
0807 19:44:43.320074 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.320087 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.321081 (+ 994us) server_negotiation.cc:407] Connection header received
0807 19:44:43.321215 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.321226 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.321494 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.321710 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:43.322663 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.324449 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.326801 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.328736 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.328906 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.329161 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.329896 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.330103 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":1341,"threads_started":1}
I20260807 19:44:43.330960 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.317658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.0.198.194:44949)
0807 19:44:43.319261 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.319276 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.322317 (+ 3041us) server_negotiation.cc:407] Connection header received
0807 19:44:43.322458 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.322465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.322743 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.322954 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:43.323550 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.325106 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.327637 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.329629 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.329944 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.330195 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.330460 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.330650 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":1210,"threads_started":1}
I20260807 19:44:43.330965 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.318059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:44639 (local address 127.0.0.1:41088)
0807 19:44:43.320693 (+ 2634us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:43.320720 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:43.320894 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:43.321977 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:43.321982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:43.322014 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:43.322468 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.322481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.324652 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.324657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.326596 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.326611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.329319 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.329325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.329481 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.329509 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 19:44:43.329584 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 19:44:43.329714 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2455,"mutex_wait_us":4,"thread_start_us":3761,"threads_started":1}
I20260807 19:44:43.331002 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.317445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.0.1:49164)
0807 19:44:43.322141 (+ 4696us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:43.322166 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:43.322304 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:43.322934 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:43.322939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:43.322965 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:43.323317 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.323326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.325304 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.325309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.327468 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.327480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.330020 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.330028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.330239 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.330280 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:44:43.330384 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:44:43.330531 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4548,"mutex_wait_us":679,"thread_start_us":2166,"threads_started":1}
I20260807 19:44:43.331996 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.317896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.0.198.195:35371)
0807 19:44:43.320407 (+ 2511us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.320419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.320455 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:44:43.320578 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.320596 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.320903 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.322066 (+ 1163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:43.322323 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.323967 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.327020 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.328727 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.331103 (+ 2376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.331423 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.331507 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.331688 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2340,"thread_start_us":2202,"threads_started":1}
I20260807 19:44:43.335393 9165 tablet_service.cc:1467] Processing CreateTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 (DEFAULT_TABLE table=test-table [id=7af6355935f64741934a869d393db80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.335391 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 (DEFAULT_TABLE table=test-table [id=7af6355935f64741934a869d393db80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.335335 9315 tablet_service.cc:1467] Processing CreateTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 (DEFAULT_TABLE table=test-table [id=7af6355935f64741934a869d393db80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.337085 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df1f58d6b84f12a69cd1dee78843e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.337082 9165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df1f58d6b84f12a69cd1dee78843e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.337080 9315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df1f58d6b84f12a69cd1dee78843e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.350077 9415 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Bootstrap starting.
I20260807 19:44:43.354236 9416 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Bootstrap starting.
I20260807 19:44:43.354604 9415 tablet_bootstrap.cc:654] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.355861 9417 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Bootstrap starting.
I20260807 19:44:43.356963 9415 log.cc:824] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:43.359802 9416 tablet_bootstrap.cc:654] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.360044 9417 tablet_bootstrap.cc:654] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.361457 9417 log.cc:824] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:43.361653 9416 log.cc:824] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:43.362056 9415 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: No bootstrap required, opened a new log
I20260807 19:44:43.362543 9415 ts_tablet_manager.cc:1397] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 19:44:43.372223 9416 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: No bootstrap required, opened a new log
I20260807 19:44:43.372474 9417 tablet_bootstrap.cc:492] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: No bootstrap required, opened a new log
I20260807 19:44:43.372679 9416 ts_tablet_manager.cc:1397] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.005s
I20260807 19:44:43.372833 9417 ts_tablet_manager.cc:1397] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260807 19:44:43.379616 9415 raft_consensus.cc:348] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.380193 9415 raft_consensus.cc:374] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.380343 9415 raft_consensus.cc:729] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.381122 9415 consensus_queue.cc:260] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.383416 9417 raft_consensus.cc:348] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.384508 9417 raft_consensus.cc:374] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.384619 9415 ts_tablet_manager.cc:1428] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Time spent starting tablet: real 0.022s user 0.017s sys 0.000s
I20260807 19:44:43.384691 9417 raft_consensus.cc:729] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.384984 9416 raft_consensus.cc:348] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.385483 9416 raft_consensus.cc:374] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.385620 9416 raft_consensus.cc:729] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.385592 9417 consensus_queue.cc:260] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.386246 9416 consensus_queue.cc:260] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.391003 9417 ts_tablet_manager.cc:1428] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
I20260807 19:44:43.393060 9416 ts_tablet_manager.cc:1428] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
W20260807 19:44:43.476401 9390 tablet.cc:2369] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:43.517915 9422 raft_consensus.cc:482] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:43.518261 9422 raft_consensus.cc:504] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.520365 9422 leader_election.cc:288] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949), 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639)
I20260807 19:44:43.529505 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.520986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:40273 (local address 127.0.198.193:44639)
0807 19:44:43.521251 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.521258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.522839 (+ 1581us) server_negotiation.cc:407] Connection header received
0807 19:44:43.523008 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.523013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.523106 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.523273 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:43.524209 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.525296 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.527457 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.528359 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.528506 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.528565 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.529065 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.529238 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:44:43.529685 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.521014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.198.195:42695)
0807 19:44:43.522210 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:43.522241 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:43.522468 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:43.522961 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:43.522966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:43.523001 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:43.523566 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.523579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.524825 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.524832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.526798 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.526815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.528776 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.528782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.528908 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.528932 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:44:43.529061 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:44:43.529208 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":55,"thread_start_us":721,"threads_started":1}
I20260807 19:44:43.529757 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.521790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:44639 (local address 127.0.198.195:40273)
0807 19:44:43.522700 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:43.522719 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:43.522862 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:43.523324 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:43.523327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:43.523418 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:43.523938 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.523953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.525483 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.525487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.527178 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:43.527196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.528633 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.528641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:43.528867 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.528897 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:44:43.528987 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 19:44:43.529177 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":3,"thread_start_us":514,"threads_started":1}
I20260807 19:44:43.529758 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.521190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:42695 (local address 127.0.198.194:44949)
0807 19:44:43.521415 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.521423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.522392 (+ 969us) server_negotiation.cc:407] Connection header received
0807 19:44:43.522699 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.522707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.522811 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.522917 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:43.523755 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.524653 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.527112 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.528509 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.528726 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.528809 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.529122 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.529269 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:44:43.531477 9093 tablet.cc:2369] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:43.531153 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df1f58d6b84f12a69cd1dee78843e2" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409" is_pre_election: true
I20260807 19:44:43.531581 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df1f58d6b84f12a69cd1dee78843e2" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67" is_pre_election: true
I20260807 19:44:43.532083 9186 raft_consensus.cc:2454] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 0.
I20260807 19:44:43.532168 9037 raft_consensus.cc:2454] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 0.
I20260807 19:44:43.533072 9265 leader_election.cc:302] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67, ecfe78da89dd46d9a32b072d34dbf605; no voters:
I20260807 19:44:43.533712 9422 raft_consensus.cc:2781] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:43.533931 9422 raft_consensus.cc:482] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:43.534051 9422 raft_consensus.cc:3037] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:43.537352 9422 raft_consensus.cc:504] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.538183 9422 leader_election.cc:288] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 election: Requested vote from peers 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949), 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639)
I20260807 19:44:43.538820 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df1f58d6b84f12a69cd1dee78843e2" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409"
I20260807 19:44:43.538990 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df1f58d6b84f12a69cd1dee78843e2" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67"
I20260807 19:44:43.539131 9186 raft_consensus.cc:3037] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:43.539276 9037 raft_consensus.cc:3037] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:43.541777 9037 raft_consensus.cc:2454] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 1.
I20260807 19:44:43.541786 9186 raft_consensus.cc:2454] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 1.
I20260807 19:44:43.542460 9262 leader_election.cc:302] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8059002e5dc24dddbf6832eef87ace67, ecfe78da89dd46d9a32b072d34dbf605; no voters:
I20260807 19:44:43.542889 9422 raft_consensus.cc:2781] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:43.543674 9422 raft_consensus.cc:686] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 LEADER]: Becoming Leader. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Running, Role: LEADER
I20260807 19:44:43.545634 9422 consensus_queue.cc:237] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } }
I20260807 19:44:43.551754 8869 catalog_manager.cc:5495] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195). New cstate: current_term: 1 leader_uuid: "ecfe78da89dd46d9a32b072d34dbf605" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:43.582844 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:43.576022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.0.198.195:35371)
0807 19:44:43.576305 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:43.576312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:43.576445 (+ 133us) server_negotiation.cc:407] Connection header received
0807 19:44:43.576743 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:43.576747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:43.576893 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:43.577069 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:43.577465 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.578565 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.580265 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:43.580817 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:43.580928 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:43.581412 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:43.582154 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:43.582297 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:43.582413 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 19:44:43.582491 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 19:44:43.592374 9186 raft_consensus.cc:1261] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Refusing update from remote peer ecfe78da89dd46d9a32b072d34dbf605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:43.592383 9037 raft_consensus.cc:1261] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Refusing update from remote peer ecfe78da89dd46d9a32b072d34dbf605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:43.593593 9427 consensus_queue.cc:1035] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:43.594062 9422 consensus_queue.cc:1035] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:43.605165 9431 mvcc.cc:205] Tried to move back new op lower bound from 7315996195183091712 to 7315996195008782336. Current Snapshot: MvccSnapshot[applied={T|T < 7315996195183091712}]
I20260807 19:44:43.609412 9433 mvcc.cc:205] Tried to move back new op lower bound from 7315996195183091712 to 7315996195008782336. Current Snapshot: MvccSnapshot[applied={T|T < 7315996195183091712}]
I20260807 19:44:43.621587 8869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:43.621945 8869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:43.627753 8869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Sending DeleteTablet for 3 replicas of tablet 63df1f58d6b84f12a69cd1dee78843e2
I20260807 19:44:43.629428 9315 tablet_service.cc:1514] Processing DeleteTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:43 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60352
I20260807 19:44:43.629510 9016 tablet_service.cc:1514] Processing DeleteTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:43 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260807 19:44:43.630357 9165 tablet_service.cc:1514] Processing DeleteTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:43 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:49164
I20260807 19:44:43.631664 9017 tablet_service.cc:1514] Processing DeleteTablet for tablet 63df1f58d6b84f12a69cd1dee78843e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:43 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260807 19:44:43.633605 8869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:43.634672 8869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:43.646713 9443 tablet_replica.cc:330] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: stopping tablet replica
I20260807 19:44:43.647588 9443 raft_consensus.cc:2229] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:43.648355 9443 raft_consensus.cc:2258] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:43.649482 9442 tablet_replica.cc:330] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: stopping tablet replica
I20260807 19:44:43.650159 9442 raft_consensus.cc:2229] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:43.651400 9443 ts_tablet_manager.cc:1905] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:43.651767 9314 tablet_service.cc:1467] Processing CreateTablet for tablet 99f107b1ff4746fab9d869f16d164f21 (DEFAULT_TABLE table=test-table [id=c5a2949d77ca410e8bd17307ea94d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.652026 9165 tablet_service.cc:1467] Processing CreateTablet for tablet 99f107b1ff4746fab9d869f16d164f21 (DEFAULT_TABLE table=test-table [id=c5a2949d77ca410e8bd17307ea94d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.652367 9314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f107b1ff4746fab9d869f16d164f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.652599 9165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f107b1ff4746fab9d869f16d164f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.657383 9016 tablet_service.cc:1467] Processing CreateTablet for tablet 99f107b1ff4746fab9d869f16d164f21 (DEFAULT_TABLE table=test-table [id=c5a2949d77ca410e8bd17307ea94d636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:43.658061 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f107b1ff4746fab9d869f16d164f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:43.659329 9415 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Bootstrap starting.
I20260807 19:44:43.661123 9415 tablet_bootstrap.cc:654] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.662210 9442 raft_consensus.cc:2258] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:43.662350 9441 tablet_replica.cc:330] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: stopping tablet replica
I20260807 19:44:43.662601 9416 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Bootstrap starting.
I20260807 19:44:43.662881 9441 raft_consensus.cc:2229] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:43.664587 9416 tablet_bootstrap.cc:654] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.663700 9441 raft_consensus.cc:2258] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:43.667577 9443 ts_tablet_manager.cc:1918] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:43.667830 9443 log.cc:1196] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/63df1f58d6b84f12a69cd1dee78843e2
I20260807 19:44:43.667982 9441 ts_tablet_manager.cc:1905] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:43.668445 9443 ts_tablet_manager.cc:1939] T 63df1f58d6b84f12a69cd1dee78843e2 P 38a92ec4230743f5b8061e8e7525e409: Deleting consensus metadata
I20260807 19:44:43.670581 8858 catalog_manager.cc:4841] TS 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949): tablet 63df1f58d6b84f12a69cd1dee78843e2 (table test-table [id=7af6355935f64741934a869d393db80c]) successfully deleted
I20260807 19:44:43.672958 9417 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Bootstrap starting.
I20260807 19:44:43.673653 9416 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: No bootstrap required, opened a new log
I20260807 19:44:43.673934 9416 ts_tablet_manager.cc:1397] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260807 19:44:43.674953 9417 tablet_bootstrap.cc:654] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:43.676409 9416 raft_consensus.cc:348] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.676796 9416 raft_consensus.cc:374] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.677011 9416 raft_consensus.cc:729] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.677451 9416 consensus_queue.cc:260] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.679131 9416 ts_tablet_manager.cc:1428] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260807 19:44:43.679934 9441 ts_tablet_manager.cc:1918] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:43.680171 9441 log.cc:1196] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/63df1f58d6b84f12a69cd1dee78843e2
I20260807 19:44:43.680912 9441 ts_tablet_manager.cc:1939] T 63df1f58d6b84f12a69cd1dee78843e2 P ecfe78da89dd46d9a32b072d34dbf605: Deleting consensus metadata
I20260807 19:44:43.682948 9417 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: No bootstrap required, opened a new log
I20260807 19:44:43.683131 9417 ts_tablet_manager.cc:1397] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 19:44:43.683557 9415 tablet_bootstrap.cc:492] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: No bootstrap required, opened a new log
I20260807 19:44:43.683735 9415 ts_tablet_manager.cc:1397] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.009s
I20260807 19:44:43.684155 9442 ts_tablet_manager.cc:1905] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:43.684103 9417 raft_consensus.cc:348] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.684394 9417 raft_consensus.cc:374] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.684514 9417 raft_consensus.cc:729] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.684684 9415 raft_consensus.cc:348] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.684942 9415 raft_consensus.cc:374] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:43.684832 9417 consensus_queue.cc:260] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.685067 9415 raft_consensus.cc:729] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Initialized, Role: FOLLOWER
I20260807 19:44:43.685511 9417 ts_tablet_manager.cc:1428] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:44:43.686561 9415 consensus_queue.cc:260] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:43.687481 9415 ts_tablet_manager.cc:1428] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:44:43.689698 8855 catalog_manager.cc:4841] TS ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371): tablet 63df1f58d6b84f12a69cd1dee78843e2 (table test-table [id=7af6355935f64741934a869d393db80c]) successfully deleted
I20260807 19:44:43.696355 9442 ts_tablet_manager.cc:1918] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:43.696597 9442 log.cc:1196] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/63df1f58d6b84f12a69cd1dee78843e2
I20260807 19:44:43.697302 9442 ts_tablet_manager.cc:1939] T 63df1f58d6b84f12a69cd1dee78843e2 P 8059002e5dc24dddbf6832eef87ace67: Deleting consensus metadata
I20260807 19:44:43.699579 8856 catalog_manager.cc:4841] TS 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639): tablet 63df1f58d6b84f12a69cd1dee78843e2 (table test-table [id=7af6355935f64741934a869d393db80c]) successfully deleted
W20260807 19:44:43.700765 8856 catalog_manager.cc:4808] TS 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639): delete failed for tablet 63df1f58d6b84f12a69cd1dee78843e2 because the tablet was not found. No further retry: Not found: Tablet not found: 63df1f58d6b84f12a69cd1dee78843e2
W20260807 19:44:43.868139 9241 tablet.cc:2369] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:44.002925 9424 raft_consensus.cc:482] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:44.003304 9424 raft_consensus.cc:504] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.005745 9424 leader_election.cc:288] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:44.014086 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.006576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41635 (local address 127.0.198.193:44639)
0807 19:44:44.006802 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.006810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.006942 (+ 132us) server_negotiation.cc:407] Connection header received
0807 19:44:44.007294 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.007302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.007420 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.007495 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:44.008303 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.009280 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.011662 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.012994 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.013190 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.013284 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.013656 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.013808 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:44:44.014199 9447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.006124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:44639 (local address 127.0.198.194:41635)
0807 19:44:44.006830 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:44.006853 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:44.007072 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:44.007738 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:44.007742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:44.007763 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:44.008088 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.008097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.009437 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.009442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.011465 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.011481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.013274 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.013283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.013470 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.013496 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:44:44.013622 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:44:44.013777 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":1,"thread_start_us":750,"threads_started":1}
I20260807 19:44:44.014441 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.006387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:34223 (local address 127.0.198.195:35371)
0807 19:44:44.006809 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.006817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.007728 (+ 911us) server_negotiation.cc:407] Connection header received
0807 19:44:44.007902 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.007909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.008053 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.008182 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:44.008862 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.010107 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.012275 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.013132 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.013299 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.013379 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.013927 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.014085 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260807 19:44:44.015209 9448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.006112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:35371 (local address 127.0.198.194:34223)
0807 19:44:44.007597 (+ 1485us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:44.007618 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:44.007750 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:44.008219 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:44.008224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:44.008257 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:44.008624 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.008638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.010288 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.010294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.012102 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.012113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.013482 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.013490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.013711 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.013751 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:44:44.014648 (+ 897us) client_negotiation.cc:240] Negotiation successful
0807 19:44:44.014808 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"mutex_wait_us":77,"thread_start_us":1013,"threads_started":1}
I20260807 19:44:44.015216 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f107b1ff4746fab9d869f16d164f21" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67" is_pre_election: true
I20260807 19:44:44.015540 9037 raft_consensus.cc:2454] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 0.
I20260807 19:44:44.016130 9113 leader_election.cc:302] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67; no voters:
I20260807 19:44:44.016304 9337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f107b1ff4746fab9d869f16d164f21" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605" is_pre_election: true
I20260807 19:44:44.016646 9424 raft_consensus.cc:2781] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:44.016774 9424 raft_consensus.cc:482] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:44.016887 9424 raft_consensus.cc:3037] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.016873 9337 raft_consensus.cc:2454] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 0.
I20260807 19:44:44.020538 9424 raft_consensus.cc:504] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.021483 9424 leader_election.cc:288] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 election: Requested vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:44.022141 9337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f107b1ff4746fab9d869f16d164f21" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605"
I20260807 19:44:44.022413 9337 raft_consensus.cc:3037] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.022308 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f107b1ff4746fab9d869f16d164f21" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67"
I20260807 19:44:44.022598 9037 raft_consensus.cc:3037] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.024865 9037 raft_consensus.cc:2454] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 1.
I20260807 19:44:44.025425 9113 leader_election.cc:302] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67; no voters:
I20260807 19:44:44.025795 9424 raft_consensus.cc:2781] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:44.026108 9337 raft_consensus.cc:2454] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 1.
I20260807 19:44:44.026417 9424 raft_consensus.cc:686] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 1 LEADER]: Becoming Leader. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Running, Role: LEADER
I20260807 19:44:44.026870 9424 consensus_queue.cc:237] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.036056 8871 catalog_manager.cc:5495] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:44:44 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131879, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.254@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:44.157892 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.142530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:45513 (local address 127.0.0.1:41722)
0807 19:44:44.143388 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:44.143416 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:44.143618 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:44.143983 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:44.143987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:44.144752 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:44.144979 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.144990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.146184 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.146190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.147130 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.147141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.147988 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.147994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.148262 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.149028 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:44.149052 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:44.151650 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:44.154986 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:44.155005 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:44.155014 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:44.155747 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:44.156145 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:44.156151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:44.156155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:44.156319 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:44.156875 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:44.156885 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:44.157068 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 19:44:44.157262 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 19:44:44.157579 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":437,"threads_started":1}
I20260807 19:44:44.158299 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.142599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.0.198.254:45513)
0807 19:44:44.143666 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.143684 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.143709 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:44:44.143769 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.143773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.143838 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.143956 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:44.145141 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.146006 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.147298 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.147801 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.147937 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.151886 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:44.151891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:44.151894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:44.151928 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:44.154819 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:44.155880 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:44.155887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:44.155893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:44.156026 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:44.156437 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:44.156442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:44.156445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:44.156735 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:44.156920 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.157556 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.157789 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":720,"threads_started":1}
I20260807 19:44:44.163944 8871 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41722:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:44:44.165093 8871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:44.172394 9016 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb825d982c453e91ecb795b5d3ec69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6ea9d6eeb24420b9e481aac3b8db677]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:44:44.172559 9165 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb825d982c453e91ecb795b5d3ec69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6ea9d6eeb24420b9e481aac3b8db677]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:44:44.172869 9315 tablet_service.cc:1467] Processing CreateTablet for tablet 79bb825d982c453e91ecb795b5d3ec69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6ea9d6eeb24420b9e481aac3b8db677]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:44:44.173077 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb825d982c453e91ecb795b5d3ec69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.173149 9165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb825d982c453e91ecb795b5d3ec69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.173400 9315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bb825d982c453e91ecb795b5d3ec69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.179412 9416 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: Bootstrap starting.
I20260807 19:44:44.180583 9415 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: Bootstrap starting.
I20260807 19:44:44.180994 9417 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: Bootstrap starting.
I20260807 19:44:44.181126 9416 tablet_bootstrap.cc:654] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.182577 9415 tablet_bootstrap.cc:654] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.182780 9417 tablet_bootstrap.cc:654] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.185496 9415 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: No bootstrap required, opened a new log
I20260807 19:44:44.185727 9415 ts_tablet_manager.cc:1397] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260807 19:44:44.186734 9415 raft_consensus.cc:348] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.187099 9415 raft_consensus.cc:374] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.187184 9415 raft_consensus.cc:729] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.187740 9415 consensus_queue.cc:260] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.191987 9415 ts_tablet_manager.cc:1428] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 19:44:44.192982 9416 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: No bootstrap required, opened a new log
I20260807 19:44:44.193054 9417 tablet_bootstrap.cc:492] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: No bootstrap required, opened a new log
I20260807 19:44:44.193189 9416 ts_tablet_manager.cc:1397] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260807 19:44:44.193256 9417 ts_tablet_manager.cc:1397] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260807 19:44:44.193904 9417 raft_consensus.cc:348] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.194025 9416 raft_consensus.cc:348] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.194209 9417 raft_consensus.cc:374] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.194294 9417 raft_consensus.cc:729] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.194324 9416 raft_consensus.cc:374] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.194403 9416 raft_consensus.cc:729] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.194571 9417 consensus_queue.cc:260] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.194643 9416 consensus_queue.cc:260] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.195097 9424 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.195458 9424 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.195464 9417 ts_tablet_manager.cc:1428] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 19:44:44.195907 9463 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.196291 9463 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.196580 9416 ts_tablet_manager.cc:1428] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 19:44:44.196578 9462 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.197012 9462 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.323315 9424 raft_consensus.cc:482] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:44.323602 9424 raft_consensus.cc:504] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.324453 9424 leader_election.cc:288] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:44.325093 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb825d982c453e91ecb795b5d3ec69" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67" is_pre_election: true
I20260807 19:44:44.325117 9337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb825d982c453e91ecb795b5d3ec69" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605" is_pre_election: true
I20260807 19:44:44.325378 9337 raft_consensus.cc:2454] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 0.
I20260807 19:44:44.325385 9037 raft_consensus.cc:2454] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 0.
I20260807 19:44:44.325918 9113 leader_election.cc:302] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67, ecfe78da89dd46d9a32b072d34dbf605; no voters:
I20260807 19:44:44.326260 9424 raft_consensus.cc:2781] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:44.326357 9424 raft_consensus.cc:482] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:44.326469 9424 raft_consensus.cc:3037] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.328898 9424 raft_consensus.cc:504] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.329578 9424 leader_election.cc:288] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 election: Requested vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:44.330077 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb825d982c453e91ecb795b5d3ec69" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67"
I20260807 19:44:44.330169 9337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bb825d982c453e91ecb795b5d3ec69" candidate_uuid: "38a92ec4230743f5b8061e8e7525e409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605"
I20260807 19:44:44.330399 9037 raft_consensus.cc:3037] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.330458 9337 raft_consensus.cc:3037] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.333040 9037 raft_consensus.cc:2454] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 1.
I20260807 19:44:44.333144 9337 raft_consensus.cc:2454] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a92ec4230743f5b8061e8e7525e409 in term 1.
I20260807 19:44:44.333547 9113 leader_election.cc:302] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67; no voters:
I20260807 19:44:44.333920 9424 raft_consensus.cc:2781] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:44.334129 9424 raft_consensus.cc:686] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [term 1 LEADER]: Becoming Leader. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Running, Role: LEADER
I20260807 19:44:44.334419 9424 consensus_queue.cc:237] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.336306 9449 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: TxnStatusTablet state changed. Reason: New leader 38a92ec4230743f5b8061e8e7525e409. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.336701 9449 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:44:44.337541 9465 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:44:44.337843 9465 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:44:44.338201 8871 catalog_manager.cc:5495] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:44.390556 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.383242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.0.198.194:44949)
0807 19:44:44.384420 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.384425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.384447 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:44:44.384572 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.384576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.384697 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.384866 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:44.385303 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.386164 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.387797 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.388360 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.388516 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.389118 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:44.389788 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:44.389965 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.390070 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.390163 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":779,"threads_started":1}
I20260807 19:44:44.390533 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.383126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.0.1:49172)
0807 19:44:44.383613 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:44.383669 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:44.383926 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:44.384884 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:44.384889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:44.384902 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:44:44.385131 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.385143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.386305 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.386313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.387664 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.387680 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.388544 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.388550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.388800 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.388874 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:44:44.389908 (+ 1034us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:44:44.389935 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:44:44.390029 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 19:44:44.390213 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263}
I20260807 19:44:44.395951 9037 raft_consensus.cc:1261] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Refusing update from remote peer 38a92ec4230743f5b8061e8e7525e409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:44.396230 9337 raft_consensus.cc:1261] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Refusing update from remote peer 38a92ec4230743f5b8061e8e7525e409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:44.396842 9424 consensus_queue.cc:1035] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:44.397199 9449 consensus_queue.cc:1035] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:44.405292 9462 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: TxnStatusTablet state changed. Reason: New leader 38a92ec4230743f5b8061e8e7525e409. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.405644 9462 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.410408 9424 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.410810 9424 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:44:44.413740 9462 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.414045 9462 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P ecfe78da89dd46d9a32b072d34dbf605: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.417264 9463 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: TxnStatusTablet state changed. Reason: New leader 38a92ec4230743f5b8061e8e7525e409. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.417605 9463 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.419660 9449 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.419997 9449 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 38a92ec4230743f5b8061e8e7525e409: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:44:44.422339 9471 mvcc.cc:205] Tried to move back new op lower bound from 7315996198477578240 to 7315996198236241920. Current Snapshot: MvccSnapshot[applied={T|T < 7315996198477578240}]
I20260807 19:44:44.425655 9463 tablet_replica.cc:439] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38a92ec4230743f5b8061e8e7525e409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } } }
I20260807 19:44:44.425994 9463 tablet_replica.cc:442] T 79bb825d982c453e91ecb795b5d3ec69 P 8059002e5dc24dddbf6832eef87ace67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:44:44.428414 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.419555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.0.198.194:44949)
0807 19:44:44.419778 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.419789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.420433 (+ 644us) server_negotiation.cc:407] Connection header received
0807 19:44:44.420650 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.420656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.420736 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.420839 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:44.421624 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.423090 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.425896 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.426582 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.426693 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.427437 (+ 744us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:44.427579 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:44.427648 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.427929 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.428030 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:44:44.449990 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.441782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41681 (local address 127.0.198.194:44949)
0807 19:44:44.442046 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.442055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.442891 (+ 836us) server_negotiation.cc:407] Connection header received
0807 19:44:44.443098 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.443104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.443223 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.443322 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:44.443987 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.445211 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.447176 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.448165 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.448347 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.448912 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:44.449165 (+ 253us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:44.449282 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.449470 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.449574 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":70}
I20260807 19:44:44.450027 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.441826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.198.194:41681)
0807 19:44:44.442775 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:44.442793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:44.442941 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:44.443396 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:44.443403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:44.443422 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:44:44.443836 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.443846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.445380 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.445384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.446980 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:44.446996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.448445 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.448450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:44.448583 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.448684 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:44:44.449300 (+ 616us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:44:44.449325 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:44:44.449404 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 19:44:44.449536 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":22,"thread_start_us":502,"threads_started":1}
I20260807 19:44:44.461064 9337 raft_consensus.cc:1261] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Refusing update from remote peer 38a92ec4230743f5b8061e8e7525e409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:44.462025 9036 raft_consensus.cc:1261] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Refusing update from remote peer 38a92ec4230743f5b8061e8e7525e409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:44.461968 9483 consensus_queue.cc:1035] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:44.462649 9483 consensus_queue.cc:1035] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:44.535588 8870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:44.535928 8870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:44.539691 8870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Sending DeleteTablet for 3 replicas of tablet 99f107b1ff4746fab9d869f16d164f21
I20260807 19:44:44.540803 9016 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f107b1ff4746fab9d869f16d164f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:44 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260807 19:44:44.540939 9164 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f107b1ff4746fab9d869f16d164f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:44 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:49164
I20260807 19:44:44.540977 9315 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f107b1ff4746fab9d869f16d164f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:44 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60352
I20260807 19:44:44.556473 9488 tablet_replica.cc:330] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: stopping tablet replica
I20260807 19:44:44.556855 9488 raft_consensus.cc:2229] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:44.558609 9488 raft_consensus.cc:2258] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:44.560720 9488 ts_tablet_manager.cc:1905] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:44.561360 9489 tablet_replica.cc:330] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: stopping tablet replica
I20260807 19:44:44.561641 9489 raft_consensus.cc:2229] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:44.561947 9489 raft_consensus.cc:2258] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:44.564016 9490 tablet_replica.cc:330] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: stopping tablet replica
I20260807 19:44:44.564299 9490 raft_consensus.cc:2229] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:44.564529 9489 ts_tablet_manager.cc:1905] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:44.565006 9490 raft_consensus.cc:2258] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:44.566339 9490 ts_tablet_manager.cc:1905] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:44.570914 9488 ts_tablet_manager.cc:1918] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:44.571190 9488 log.cc:1196] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/99f107b1ff4746fab9d869f16d164f21
I20260807 19:44:44.571903 9489 ts_tablet_manager.cc:1918] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:44.571935 9488 ts_tablet_manager.cc:1939] T 99f107b1ff4746fab9d869f16d164f21 P 8059002e5dc24dddbf6832eef87ace67: Deleting consensus metadata
I20260807 19:44:44.572062 9489 log.cc:1196] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/99f107b1ff4746fab9d869f16d164f21
I20260807 19:44:44.572546 9489 ts_tablet_manager.cc:1939] T 99f107b1ff4746fab9d869f16d164f21 P ecfe78da89dd46d9a32b072d34dbf605: Deleting consensus metadata
I20260807 19:44:44.573644 8856 catalog_manager.cc:4841] TS 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639): tablet 99f107b1ff4746fab9d869f16d164f21 (table test-table [id=c5a2949d77ca410e8bd17307ea94d636]) successfully deleted
I20260807 19:44:44.575552 9490 ts_tablet_manager.cc:1918] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:44.575788 9490 log.cc:1196] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/99f107b1ff4746fab9d869f16d164f21
I20260807 19:44:44.576033 8855 catalog_manager.cc:4841] TS ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371): tablet 99f107b1ff4746fab9d869f16d164f21 (table test-table [id=c5a2949d77ca410e8bd17307ea94d636]) successfully deleted
I20260807 19:44:44.576383 9490 ts_tablet_manager.cc:1939] T 99f107b1ff4746fab9d869f16d164f21 P 38a92ec4230743f5b8061e8e7525e409: Deleting consensus metadata
I20260807 19:44:44.577637 8858 catalog_manager.cc:4841] TS 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949): tablet 99f107b1ff4746fab9d869f16d164f21 (table test-table [id=c5a2949d77ca410e8bd17307ea94d636]) successfully deleted
Aug 07 19:44:44 dist-test-slave-btw8 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:44:44 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:44 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:44:44.643489 9458 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:44.630855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.0.198.254:45513)
0807 19:44:44.631094 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.631102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.631575 (+ 473us) server_negotiation.cc:407] Connection header received
0807 19:44:44.631772 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.631780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.631887 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.632018 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:44.633590 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.634827 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.636396 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.637062 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.637220 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.643135 (+ 5915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41738: BlockingRecv error: recv got EOF from 127.0.0.1:41738 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 19:44:44 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM
I20260807 19:44:44.663555 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:44.647243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.0.198.254:45513)
0807 19:44:44.647529 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:44.647536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:44.647864 (+ 328us) server_negotiation.cc:407] Connection header received
0807 19:44:44.648114 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:44.648119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:44.648188 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:44.648301 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:44.649963 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.650808 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.652334 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:44.652796 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:44.652958 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:44.657457 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:44.657463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:44.657467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:44.657514 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:44.660516 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:44.661315 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:44.661320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:44.661322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:44.661460 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:44.661873 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:44.661877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:44.661879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:44.662240 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:44.662561 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:44.662899 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:44:44.663164 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:44:44.667537 8870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:44.668284 8870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:44.676367 9016 tablet_service.cc:1467] Processing CreateTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 (DEFAULT_TABLE table=test-table [id=b6a175116181455bb787c73482f404e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:44.676478 9315 tablet_service.cc:1467] Processing CreateTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 (DEFAULT_TABLE table=test-table [id=b6a175116181455bb787c73482f404e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:44.676913 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9758fb7c4a4aebae20a9f22c0ff186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.676970 9315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9758fb7c4a4aebae20a9f22c0ff186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.680195 9164 tablet_service.cc:1467] Processing CreateTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 (DEFAULT_TABLE table=test-table [id=b6a175116181455bb787c73482f404e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:44.680862 9164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9758fb7c4a4aebae20a9f22c0ff186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:44.684048 9417 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Bootstrap starting.
I20260807 19:44:44.685877 9417 tablet_bootstrap.cc:654] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.690011 9415 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Bootstrap starting.
I20260807 19:44:44.691709 9416 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Bootstrap starting.
I20260807 19:44:44.691982 9415 tablet_bootstrap.cc:654] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.693708 9416 tablet_bootstrap.cc:654] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:44.694115 9415 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: No bootstrap required, opened a new log
I20260807 19:44:44.694247 9415 ts_tablet_manager.cc:1397] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:44.694940 9415 raft_consensus.cc:348] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.695358 9415 raft_consensus.cc:374] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.695811 9415 raft_consensus.cc:729] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.696193 9417 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: No bootstrap required, opened a new log
I20260807 19:44:44.696406 9417 ts_tablet_manager.cc:1397] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:44:44.696692 9415 consensus_queue.cc:260] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.697418 9417 raft_consensus.cc:348] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.697688 9417 raft_consensus.cc:374] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.697793 9417 raft_consensus.cc:729] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.698024 9415 ts_tablet_manager.cc:1428] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 19:44:44.698565 9417 consensus_queue.cc:260] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.699194 9416 tablet_bootstrap.cc:492] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: No bootstrap required, opened a new log
I20260807 19:44:44.699406 9416 ts_tablet_manager.cc:1397] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260807 19:44:44.699542 9417 ts_tablet_manager.cc:1428] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:44:44.700311 9416 raft_consensus.cc:348] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.700620 9416 raft_consensus.cc:374] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:44.700742 9416 raft_consensus.cc:729] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Initialized, Role: FOLLOWER
I20260807 19:44:44.701045 9416 consensus_queue.cc:260] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.701761 9416 ts_tablet_manager.cc:1428] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 19:44:44.972290 9462 raft_consensus.cc:482] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:44.972631 9462 raft_consensus.cc:504] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.973580 9462 leader_election.cc:288] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949)
I20260807 19:44:44.974263 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9758fb7c4a4aebae20a9f22c0ff186" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67" is_pre_election: true
I20260807 19:44:44.974342 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9758fb7c4a4aebae20a9f22c0ff186" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409" is_pre_election: true
I20260807 19:44:44.974594 9037 raft_consensus.cc:2454] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 0.
I20260807 19:44:44.974618 9186 raft_consensus.cc:2454] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 0.
I20260807 19:44:44.975147 9265 leader_election.cc:302] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, ecfe78da89dd46d9a32b072d34dbf605; no voters:
I20260807 19:44:44.975526 9462 raft_consensus.cc:2781] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:44.975695 9462 raft_consensus.cc:482] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:44.975828 9462 raft_consensus.cc:3037] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.978721 9462 raft_consensus.cc:504] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.979579 9462 leader_election.cc:288] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 election: Requested vote from peers 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639), 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949)
I20260807 19:44:44.980149 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9758fb7c4a4aebae20a9f22c0ff186" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409"
I20260807 19:44:44.980212 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9758fb7c4a4aebae20a9f22c0ff186" candidate_uuid: "ecfe78da89dd46d9a32b072d34dbf605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8059002e5dc24dddbf6832eef87ace67"
I20260807 19:44:44.980417 9186 raft_consensus.cc:3037] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.980437 9037 raft_consensus.cc:3037] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:44.983597 9037 raft_consensus.cc:2454] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 1.
I20260807 19:44:44.984122 9262 leader_election.cc:302] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8059002e5dc24dddbf6832eef87ace67, ecfe78da89dd46d9a32b072d34dbf605; no voters:
I20260807 19:44:44.983927 9186 raft_consensus.cc:2454] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecfe78da89dd46d9a32b072d34dbf605 in term 1.
I20260807 19:44:44.984509 9462 raft_consensus.cc:2781] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:44.985133 9462 raft_consensus.cc:686] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 LEADER]: Becoming Leader. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Running, Role: LEADER
I20260807 19:44:44.985505 9462 consensus_queue.cc:237] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:44.989185 8871 catalog_manager.cc:5495] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195). New cstate: current_term: 1 leader_uuid: "ecfe78da89dd46d9a32b072d34dbf605" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } health_report { overall_health: HEALTHY } } }
I20260807 19:44:45.077723 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.070175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.0.198.195:35371)
0807 19:44:45.070809 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.070816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.070840 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:44:45.070918 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.070924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.071017 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.071138 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:45.071692 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.072896 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.075157 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.075764 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.075938 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.076598 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.076786 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.076917 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.077111 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.077219 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":355,"threads_started":1}
I20260807 19:44:45.081717 9186 raft_consensus.cc:1261] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Refusing update from remote peer ecfe78da89dd46d9a32b072d34dbf605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:45.081831 9037 raft_consensus.cc:1261] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Refusing update from remote peer ecfe78da89dd46d9a32b072d34dbf605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:45.082621 9462 consensus_queue.cc:1035] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:45.083001 9507 consensus_queue.cc:1035] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:45.120121 8871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:45.120601 8871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:45.124893 8871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Sending DeleteTablet for 3 replicas of tablet bf9758fb7c4a4aebae20a9f22c0ff186
I20260807 19:44:45.126082 9016 tablet_service.cc:1514] Processing DeleteTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260807 19:44:45.126155 9164 tablet_service.cc:1514] Processing DeleteTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:49164
I20260807 19:44:45.126660 9315 tablet_service.cc:1514] Processing DeleteTablet for tablet bf9758fb7c4a4aebae20a9f22c0ff186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60352
I20260807 19:44:45.126992 9521 tablet_replica.cc:330] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: stopping tablet replica
I20260807 19:44:45.127264 9521 raft_consensus.cc:2229] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:45.127728 9521 raft_consensus.cc:2258] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.128647 8869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:45.129426 8869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:45.129568 9521 ts_tablet_manager.cc:1905] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.140410 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 45d98cbeb535420f85f1299b3e9b242f (DEFAULT_TABLE table=test-table [id=f08479b023a94f2faca9d177022f9e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:45.140561 9164 tablet_service.cc:1467] Processing CreateTablet for tablet 45d98cbeb535420f85f1299b3e9b242f (DEFAULT_TABLE table=test-table [id=f08479b023a94f2faca9d177022f9e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:45.140611 9521 ts_tablet_manager.cc:1918] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:45.140980 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d98cbeb535420f85f1299b3e9b242f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:45.141072 9521 log.cc:1196] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/bf9758fb7c4a4aebae20a9f22c0ff186
I20260807 19:44:45.141752 9521 ts_tablet_manager.cc:1939] T bf9758fb7c4a4aebae20a9f22c0ff186 P 38a92ec4230743f5b8061e8e7525e409: Deleting consensus metadata
I20260807 19:44:45.142060 9314 tablet_service.cc:1467] Processing CreateTablet for tablet 45d98cbeb535420f85f1299b3e9b242f (DEFAULT_TABLE table=test-table [id=f08479b023a94f2faca9d177022f9e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:45.142565 9314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d98cbeb535420f85f1299b3e9b242f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:45.141108 9164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d98cbeb535420f85f1299b3e9b242f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:45.147639 8858 catalog_manager.cc:4841] TS 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949): tablet bf9758fb7c4a4aebae20a9f22c0ff186 (table test-table [id=b6a175116181455bb787c73482f404e9]) successfully deleted
I20260807 19:44:45.148168 9417 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Bootstrap starting.
I20260807 19:44:45.150112 9417 tablet_bootstrap.cc:654] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:45.153785 9416 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Bootstrap starting.
I20260807 19:44:45.155840 9415 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Bootstrap starting.
I20260807 19:44:45.157392 9416 tablet_bootstrap.cc:654] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:45.157909 9415 tablet_bootstrap.cc:654] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:45.162146 9520 tablet_replica.cc:330] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: stopping tablet replica
I20260807 19:44:45.162735 9522 tablet_replica.cc:330] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: stopping tablet replica
I20260807 19:44:45.163120 9522 raft_consensus.cc:2229] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:45.163611 9520 raft_consensus.cc:2229] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:45.164259 9522 raft_consensus.cc:2258] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.164767 9520 raft_consensus.cc:2258] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.166268 9520 ts_tablet_manager.cc:1905] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.167990 9522 ts_tablet_manager.cc:1905] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.168936 9415 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: No bootstrap required, opened a new log
I20260807 19:44:45.169160 9415 ts_tablet_manager.cc:1397] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.001s
I20260807 19:44:45.169206 9417 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: No bootstrap required, opened a new log
I20260807 19:44:45.169409 9417 ts_tablet_manager.cc:1397] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.002s
I20260807 19:44:45.170241 9415 raft_consensus.cc:348] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.170346 9417 raft_consensus.cc:348] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.170574 9415 raft_consensus.cc:374] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:45.170645 9417 raft_consensus.cc:374] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:45.170673 9415 raft_consensus.cc:729] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecfe78da89dd46d9a32b072d34dbf605, State: Initialized, Role: FOLLOWER
I20260807 19:44:45.170734 9417 raft_consensus.cc:729] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Initialized, Role: FOLLOWER
I20260807 19:44:45.170993 9415 consensus_queue.cc:260] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.171119 9417 consensus_queue.cc:260] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.171844 9415 ts_tablet_manager.cc:1428] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 19:44:45.175297 9520 ts_tablet_manager.cc:1918] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:45.175642 9520 log.cc:1196] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/bf9758fb7c4a4aebae20a9f22c0ff186
I20260807 19:44:45.176258 9520 ts_tablet_manager.cc:1939] T bf9758fb7c4a4aebae20a9f22c0ff186 P 8059002e5dc24dddbf6832eef87ace67: Deleting consensus metadata
I20260807 19:44:45.175266 9463 raft_consensus.cc:482] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:45.176738 9463 raft_consensus.cc:504] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.176993 9416 tablet_bootstrap.cc:492] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: No bootstrap required, opened a new log
I20260807 19:44:45.177228 9416 ts_tablet_manager.cc:1397] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.005s
I20260807 19:44:45.178175 9416 raft_consensus.cc:348] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.178423 9416 raft_consensus.cc:374] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:45.178515 9416 raft_consensus.cc:729] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a92ec4230743f5b8061e8e7525e409, State: Initialized, Role: FOLLOWER
I20260807 19:44:45.178792 9416 consensus_queue.cc:260] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.178886 9463 leader_election.cc:288] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:45.180572 8856 catalog_manager.cc:4841] TS 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639): tablet bf9758fb7c4a4aebae20a9f22c0ff186 (table test-table [id=b6a175116181455bb787c73482f404e9]) successfully deleted
I20260807 19:44:45.181198 9416 ts_tablet_manager.cc:1428] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260807 19:44:45.193032 9522 ts_tablet_manager.cc:1918] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:45.193281 9522 log.cc:1196] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/bf9758fb7c4a4aebae20a9f22c0ff186
I20260807 19:44:45.193890 9522 ts_tablet_manager.cc:1939] T bf9758fb7c4a4aebae20a9f22c0ff186 P ecfe78da89dd46d9a32b072d34dbf605: Deleting consensus metadata
I20260807 19:44:45.195617 8855 catalog_manager.cc:4841] TS ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371): tablet bf9758fb7c4a4aebae20a9f22c0ff186 (table test-table [id=b6a175116181455bb787c73482f404e9]) successfully deleted
I20260807 19:44:45.203444 9417 ts_tablet_manager.cc:1428] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260807 19:44:45.214237 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.180007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:46531 (local address 127.0.198.194:44949)
0807 19:44:45.204543 (+ 24536us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.204554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.207541 (+ 2987us) server_negotiation.cc:407] Connection header received
0807 19:44:45.207748 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.207752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.207815 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.207958 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:45.208966 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.210046 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.212060 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.213003 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.213255 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.213345 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.213663 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.213812 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24311,"thread_start_us":26505,"threads_started":1}
I20260807 19:44:45.214443 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.179585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.198.193:46531)
0807 19:44:45.207414 (+ 27829us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:45.207435 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:45.207593 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:45.208350 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:45.208369 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:45.208401 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:45.208800 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.208811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.210209 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.210215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.211867 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.211881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.213256 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.213263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.213467 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.213494 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:44:45.213627 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 19:44:45.213789 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27641,"mutex_wait_us":43,"thread_start_us":27044,"threads_started":1}
I20260807 19:44:45.214794 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.179288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:39151 (local address 127.0.198.195:35371)
0807 19:44:45.179500 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.179510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.205265 (+ 25755us) server_negotiation.cc:407] Connection header received
0807 19:44:45.205624 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.205633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.205772 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.205862 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:45.206933 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.208140 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.211810 (+ 3670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.213111 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.213279 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.213356 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.214070 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.214247 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:44:45.215237 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45d98cbeb535420f85f1299b3e9b242f" candidate_uuid: "8059002e5dc24dddbf6832eef87ace67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409" is_pre_election: true
I20260807 19:44:45.215543 9186 raft_consensus.cc:2454] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8059002e5dc24dddbf6832eef87ace67 in term 0.
I20260807 19:44:45.215641 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.179204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:35371 (local address 127.0.198.193:39151)
0807 19:44:45.205092 (+ 25888us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:45.205132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:45.205419 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:45.206224 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:45.206232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:45.206273 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:45.206696 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.206712 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.208360 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.208368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.211541 (+ 3173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.211561 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.213611 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.213618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.213830 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.213865 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:44:45.215006 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0807 19:44:45.215177 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25709,"mutex_wait_us":2,"thread_start_us":27859,"threads_started":1}
I20260807 19:44:45.216467 8968 leader_election.cc:302] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67; no voters:
I20260807 19:44:45.216475 9336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45d98cbeb535420f85f1299b3e9b242f" candidate_uuid: "8059002e5dc24dddbf6832eef87ace67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605" is_pre_election: true
I20260807 19:44:45.216770 9336 raft_consensus.cc:2454] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8059002e5dc24dddbf6832eef87ace67 in term 0.
I20260807 19:44:45.216943 9463 raft_consensus.cc:2781] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:45.217075 9463 raft_consensus.cc:482] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:45.217197 9463 raft_consensus.cc:3037] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:45.219699 9463 raft_consensus.cc:504] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.220464 9463 leader_election.cc:288] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [CANDIDATE]: Term 1 election: Requested vote from peers 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949), ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371)
I20260807 19:44:45.221164 9186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45d98cbeb535420f85f1299b3e9b242f" candidate_uuid: "8059002e5dc24dddbf6832eef87ace67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a92ec4230743f5b8061e8e7525e409"
I20260807 19:44:45.221228 9336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45d98cbeb535420f85f1299b3e9b242f" candidate_uuid: "8059002e5dc24dddbf6832eef87ace67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecfe78da89dd46d9a32b072d34dbf605"
I20260807 19:44:45.221431 9186 raft_consensus.cc:3037] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:45.221490 9336 raft_consensus.cc:3037] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:45.224293 9186 raft_consensus.cc:2454] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8059002e5dc24dddbf6832eef87ace67 in term 1.
I20260807 19:44:45.224812 9336 raft_consensus.cc:2454] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8059002e5dc24dddbf6832eef87ace67 in term 1.
I20260807 19:44:45.224956 8968 leader_election.cc:302] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a92ec4230743f5b8061e8e7525e409, 8059002e5dc24dddbf6832eef87ace67; no voters:
I20260807 19:44:45.225379 9463 raft_consensus.cc:2781] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:45.225948 9463 raft_consensus.cc:686] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 1 LEADER]: Becoming Leader. State: Replica: 8059002e5dc24dddbf6832eef87ace67, State: Running, Role: LEADER
I20260807 19:44:45.226413 9463 consensus_queue.cc:237] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } }
I20260807 19:44:45.230698 8871 catalog_manager.cc:5495] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "8059002e5dc24dddbf6832eef87ace67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8059002e5dc24dddbf6832eef87ace67" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 44639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 } health_report { overall_health: UNKNOWN } } }
I20260807 19:44:45.274271 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.267103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.0.198.193:44639)
0807 19:44:45.267898 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.267904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.267927 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:45.268031 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.268035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.268121 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.268337 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:45.268871 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.269824 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.271443 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.271953 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.272119 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.272778 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.273452 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.273636 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.273782 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.273888 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":441,"threads_started":1}
I20260807 19:44:45.293359 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.286661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:35031 (local address 127.0.198.194:44949)
0807 19:44:45.286896 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.286906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.287653 (+ 747us) server_negotiation.cc:407] Connection header received
0807 19:44:45.287888 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.287895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.287995 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.288109 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:45.288616 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.289833 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.291331 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.291990 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.292119 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.292622 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.292760 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.292848 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.292974 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.293061 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:44:45.293388 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.286677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44949 (local address 127.0.198.193:35031)
0807 19:44:45.287556 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:45.287574 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:45.287708 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:45.288136 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:45.288140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:45.288151 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:44:45.288474 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.288485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.289999 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.290003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.291147 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.291161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.292208 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.292214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.292358 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.292436 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.292864 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.292873 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:44:45.292938 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 19:44:45.293079 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":465,"threads_started":1}
I20260807 19:44:45.304339 9336 raft_consensus.cc:1261] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Refusing update from remote peer 8059002e5dc24dddbf6832eef87ace67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:45.304414 9186 raft_consensus.cc:1261] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Refusing update from remote peer 8059002e5dc24dddbf6832eef87ace67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:45.305214 9463 consensus_queue.cc:1035] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a92ec4230743f5b8061e8e7525e409" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:45.305557 9529 consensus_queue.cc:1035] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecfe78da89dd46d9a32b072d34dbf605" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 35371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:45.343819 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.336472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:53023 (local address 127.0.198.193:44639)
0807 19:44:45.336651 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.336657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.336786 (+ 129us) server_negotiation.cc:407] Connection header received
0807 19:44:45.337015 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.337021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.337106 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.337224 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:45.338229 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.339301 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.341670 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.342167 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.342292 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.342969 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.343109 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.343176 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.343429 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.343524 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:44:45.345013 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.335987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:44639 (local address 127.0.198.194:53023)
0807 19:44:45.336661 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:45.336693 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:45.336851 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:45.337353 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:45.337359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:45.337374 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:44:45.338028 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.338043 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.339587 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.339593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.341443 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:45.341466 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.342409 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.342416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:45.342631 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.342683 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:44:45.343262 (+ 579us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:44:45.343273 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:44:45.344445 (+ 1172us) client_negotiation.cc:240] Negotiation successful
0807 19:44:45.344605 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":685,"threads_started":1}
I20260807 19:44:45.390475 8871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:45.390755 8871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:45.394266 8871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e771646645f4a4686696e1c1f7d3231: Sending DeleteTablet for 3 replicas of tablet 45d98cbeb535420f85f1299b3e9b242f
I20260807 19:44:45.395591 9314 tablet_service.cc:1514] Processing DeleteTablet for tablet 45d98cbeb535420f85f1299b3e9b242f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:60352
I20260807 19:44:45.395921 9522 tablet_replica.cc:330] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: stopping tablet replica
I20260807 19:44:45.395872 9016 tablet_service.cc:1514] Processing DeleteTablet for tablet 45d98cbeb535420f85f1299b3e9b242f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260807 19:44:45.396180 9520 tablet_replica.cc:330] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: stopping tablet replica
I20260807 19:44:45.396138 9522 raft_consensus.cc:2229] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:45.396374 9522 raft_consensus.cc:2258] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.396389 9520 raft_consensus.cc:2229] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:45.396955 9520 raft_consensus.cc:2258] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.397619 9522 ts_tablet_manager.cc:1905] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.398584 9520 ts_tablet_manager.cc:1905] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.401132 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 8948
I20260807 19:44:45.404947 9522 ts_tablet_manager.cc:1918] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:45.405144 9522 log.cc:1196] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/45d98cbeb535420f85f1299b3e9b242f
I20260807 19:44:45.405614 9522 ts_tablet_manager.cc:1939] T 45d98cbeb535420f85f1299b3e9b242f P ecfe78da89dd46d9a32b072d34dbf605: Deleting consensus metadata
I20260807 19:44:45.407618 9164 tablet_service.cc:1514] Processing DeleteTablet for tablet 45d98cbeb535420f85f1299b3e9b242f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:45 UTC) from {username='oryx', principal='oryx/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:49164
I20260807 19:44:45.408121 9521 tablet_replica.cc:330] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: stopping tablet replica
I20260807 19:44:45.408088 8855 catalog_manager.cc:4841] TS ecfe78da89dd46d9a32b072d34dbf605 (127.0.198.195:35371): tablet 45d98cbeb535420f85f1299b3e9b242f (table test-table [id=f08479b023a94f2faca9d177022f9e18]) successfully deleted
I20260807 19:44:45.408344 9521 raft_consensus.cc:2229] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:45.408546 9521 raft_consensus.cc:2258] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:45.409749 9521 ts_tablet_manager.cc:1905] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:45.413112 9520 ts_tablet_manager.cc:1918] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:45.413323 9520 log.cc:1196] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/45d98cbeb535420f85f1299b3e9b242f
I20260807 19:44:45.413826 9520 ts_tablet_manager.cc:1939] T 45d98cbeb535420f85f1299b3e9b242f P 8059002e5dc24dddbf6832eef87ace67: Deleting consensus metadata
I20260807 19:44:45.415211 8856 catalog_manager.cc:4841] TS 8059002e5dc24dddbf6832eef87ace67 (127.0.198.193:44639): tablet 45d98cbeb535420f85f1299b3e9b242f (table test-table [id=f08479b023a94f2faca9d177022f9e18]) successfully deleted
I20260807 19:44:45.416625 9521 ts_tablet_manager.cc:1918] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:44:45.416817 9521 log.cc:1196] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/45d98cbeb535420f85f1299b3e9b242f
I20260807 19:44:45.417297 9521 ts_tablet_manager.cc:1939] T 45d98cbeb535420f85f1299b3e9b242f P 38a92ec4230743f5b8061e8e7525e409: Deleting consensus metadata
I20260807 19:44:45.418370 8858 catalog_manager.cc:4841] TS 38a92ec4230743f5b8061e8e7525e409 (127.0.198.194:44949): tablet 45d98cbeb535420f85f1299b3e9b242f (table test-table [id=f08479b023a94f2faca9d177022f9e18]) successfully deleted
Aug 07 19:44:45 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.193@KRBTEST.COM
I20260807 19:44:45.431540 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.403463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.0.198.193:44639)
0807 19:44:45.405114 (+ 1651us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.405120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.405144 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:44:45.405466 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.405475 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.405612 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.405737 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:45.409999 (+ 4262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.411249 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.413475 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.414200 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.414309 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.423876 (+ 9567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:45.423897 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:45.423914 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:45.423961 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:45.427810 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.428798 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.428807 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.428809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.428970 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.429466 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.429470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.429473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.429931 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:45.430339 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.430698 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.430997 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1502}
I20260807 19:44:45.432333 9087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:45.548965 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 8948
I20260807 19:44:45.572046 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9096
Aug 07 19:44:45 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.194@KRBTEST.COM
I20260807 19:44:45.589215 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.573085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43748 (local address 127.0.198.194:44949)
0807 19:44:45.573301 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.573306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.573324 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:44:45.573388 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.573392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.573454 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.573557 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:45.574926 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.575782 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.577166 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.577595 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.577692 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.582453 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:45.582470 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:45.582507 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:45.582546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:45.585732 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.586542 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.586550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.586552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.586704 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.587095 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.587100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.587102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.587895 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:45.588236 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.588529 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.588807 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 19:44:45.589985 9234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:45.749078 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9096
I20260807 19:44:45.780391 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9244
Aug 07 19:44:45 dist-test-slave-btw8 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.198.195@KRBTEST.COM
I20260807 19:44:45.797843 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:45.781246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.0.198.195:35371)
0807 19:44:45.781760 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:45.781765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:45.781783 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:44:45.781841 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:45.781845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:45.781916 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:45.782032 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:45.783326 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.784205 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.785511 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:45.785967 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:45.786073 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:45.791787 (+ 5714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:45.791808 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:45.791826 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:45.791875 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:45.794989 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.795955 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.795963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.795965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.796101 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:45.796467 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:45.796470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:45.796472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:45.796837 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:45.797071 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:45.797348 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:44:45.797541 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":285,"threads_started":1}
I20260807 19:44:45.798646 9384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:45.937747 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9244
I20260807 19:44:45.964200 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 8838
I20260807 19:44:45.965562 8912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:46.088893 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 8838
2026-08-07T19:44:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (8092 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:46 dist-test-slave-btw8 krb5kdc[9552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:46 dist-test-slave-btw8 krb5kdc[9552](info): set up 2 sockets
Aug 07 19:44:46 dist-test-slave-btw8 krb5kdc[9552](info): commencing operation
krb5kdc: starting...
W20260807 19:44:47.342284 795 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:47 dist-test-slave-btw8 krb5kdc[9552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:47.672135 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33309
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:48.064963 9565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:48.065375 9565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:48.065596 9565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:48.073603 9565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:48.073730 9565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:48.073802 9565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:48.073887 9565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:48.073969 9565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:48.085055 9565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:48.086526 9565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:48.088078 9565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:48.101071 9570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:48.101130 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:48.101130 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:48.102340 9565 server_base.cc:1034] running on GCE node
I20260807 19:44:48.103200 9565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:48.105154 9565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:48.106432 9565 hybrid_clock.cc:648] HybridClock initialized: now 1786131888106361 us; error 59 us; skew 500 ppm
I20260807 19:44:48.106855 9565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T19:44:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2090 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:48 dist-test-slave-btw8 krb5kdc[9580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:48 dist-test-slave-btw8 krb5kdc[9580](info): set up 2 sockets
Aug 07 19:44:48 dist-test-slave-btw8 krb5kdc[9580](info): commencing operation
krb5kdc: starting...
W20260807 19:44:49.366174 795 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:44:49 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:44:49.699093 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39291
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:50.064364 9594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:50.064700 9594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:50.064796 9594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:50.072414 9594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:50.072548 9594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:50.072597 9594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:50.072649 9594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:50.072695 9594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:50.084093 9594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:50.085563 9594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:50.086942 9594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:50.100658 9599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:50.100601 9600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:50.100579 9602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:50.101917 9594 server_base.cc:1034] running on GCE node
I20260807 19:44:50.102977 9594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:50.104709 9594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:50.105939 9594 hybrid_clock.cc:648] HybridClock initialized: now 1786131890105890 us; error 37 us; skew 500 ppm
I20260807 19:44:50.106339 9594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:50 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:50.109994 9594 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:50.112068 9594 webserver.cc:460] Webserver started at http://127.0.198.254:39399/ using document root <none> and password file <none>
I20260807 19:44:50.112851 9594 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:50.112983 9594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:50.113346 9594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:50.116914 9594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "e4160594768a4b9fb9f32bf754e4dd48"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "ae8cac50385d4fdc896552df2bcf26cb"
server_key_iv: "8d5e86eef98c0a238d20efc117483fd3"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.117928 9594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "e4160594768a4b9fb9f32bf754e4dd48"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "ae8cac50385d4fdc896552df2bcf26cb"
server_key_iv: "8d5e86eef98c0a238d20efc117483fd3"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.124131 9594 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:44:50.128556 9609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:50.130215 9594 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:44:50.130457 9594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "e4160594768a4b9fb9f32bf754e4dd48"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "ae8cac50385d4fdc896552df2bcf26cb"
server_key_iv: "8d5e86eef98c0a238d20efc117483fd3"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.130720 9594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:50.144781 9594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:50.149883 9594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:50.150230 9594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:50.172482 9594 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:39291
I20260807 19:44:50.172489 9663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:39291 every 8 connection(s)
I20260807 19:44:50.173980 9594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:50.178499 9664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:50.182807 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9594
I20260807 19:44:50.183068 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:44:50.183954 795 external_mini_cluster.cc:1428] Setting key 84a6867a127765f6a34f78f501e50ce1
I20260807 19:44:50.192915 9664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Bootstrap starting.
I20260807 19:44:50.196411 9664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:50.197571 9664 log.cc:824] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:44:50 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:50.201327 9664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: No bootstrap required, opened a new log
I20260807 19:44:50.209856 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:50.186345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.0.198.254:39291)
0807 19:44:50.187196 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:50.187208 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:50.187249 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:44:50.189036 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:50.189063 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:50.189544 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:50.190014 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:50.191810 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.193243 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.194589 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.195308 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.195610 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:50.200267 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:50.200280 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:50.200300 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:50.200377 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:50.204446 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:50.205349 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:50.205357 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:50.205365 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:50.205543 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:50.205981 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:50.205997 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:50.205999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:50.206903 (+ 904us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:50.207287 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:50.207917 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 19:44:50.208445 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":391,"threads_started":1}
I20260807 19:44:50.210373 9664 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:50.210824 9664 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:50.210955 9664 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4160594768a4b9fb9f32bf754e4dd48, State: Initialized, Role: FOLLOWER
I20260807 19:44:50.211706 9664 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:50.211999 9664 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:50.212163 9664 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:50.212365 9664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:50.215027 9664 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:50.215691 9664 leader_election.cc:302] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4160594768a4b9fb9f32bf754e4dd48; no voters:
I20260807 19:44:50.216387 9664 leader_election.cc:288] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:50.216672 9669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:50.217530 9669 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 LEADER]: Becoming Leader. State: Replica: e4160594768a4b9fb9f32bf754e4dd48, State: Running, Role: LEADER
I20260807 19:44:50.218240 9669 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:50.218997 9664 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:50.221964 9670 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4160594768a4b9fb9f32bf754e4dd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } } }
I20260807 19:44:50.222049 9671 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4160594768a4b9fb9f32bf754e4dd48. Latest consensus state: current_term: 1 leader_uuid: "e4160594768a4b9fb9f32bf754e4dd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } } }
I20260807 19:44:50.222450 9670 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:50.222465 9671 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:50.225008 9676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:50.232625 9676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:50.244408 9676 catalog_manager.cc:1347] Generated new cluster ID: 96c412f2019c4c72ab9afe6f10382770
I20260807 19:44:50.244556 9676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:50.262269 9676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:50.264443 9676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:50.274250 9676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Generated new TSK 0
I20260807 19:44:50.275472 9676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:44:50.360425 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:50.727233 9691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:50.727639 9691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:50.727746 9691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:50.735718 9691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:50.735849 9691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:50.735970 9691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:50.746639 9691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:50.748107 9691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:50.749441 9691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:50.763728 9696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:50.763825 9697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:50.764386 9699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:50.764907 9691 server_base.cc:1034] running on GCE node
I20260807 19:44:50.765537 9691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:50.766605 9691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:50.767843 9691 hybrid_clock.cc:648] HybridClock initialized: now 1786131890767791 us; error 45 us; skew 500 ppm
I20260807 19:44:50.768216 9691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:50 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:50.777318 9691 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:44:50.779021 9691 webserver.cc:460] Webserver started at http://127.0.198.193:39917/ using document root <none> and password file <none>
I20260807 19:44:50.779855 9691 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:50.779994 9691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:50.780365 9691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:50.783475 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "9265c58572674ed4a7bfd97882b54227"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "34de1cfa91ad6938b36e49a99cac2014"
server_key_iv: "4fa78888c193774f5cde9d5e320957ee"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.784350 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "9265c58572674ed4a7bfd97882b54227"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "34de1cfa91ad6938b36e49a99cac2014"
server_key_iv: "4fa78888c193774f5cde9d5e320957ee"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.789925 9691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:44:50.793967 9706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:50.795660 9691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:44:50.795887 9691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "9265c58572674ed4a7bfd97882b54227"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "34de1cfa91ad6938b36e49a99cac2014"
server_key_iv: "4fa78888c193774f5cde9d5e320957ee"
server_key_version: "encryptionkey@0"
I20260807 19:44:50.796119 9691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:50.839923 9691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:50.845669 9691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:50.846021 9691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:50.847086 9691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:50.848680 9691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:50.848786 9691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:50.848953 9691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:50.849027 9691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:50.885439 9691 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:43621
I20260807 19:44:50.885535 9824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:43621 every 8 connection(s)
I20260807 19:44:50.887027 9691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:50.894361 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9691
I20260807 19:44:50.894598 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:50.895313 795 external_mini_cluster.cc:1428] Setting key 1ef436d0bb87431299446383b6860a3e
Aug 07 19:44:50 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:50.911468 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:50.890175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:59223 (local address 127.0.198.254:39291)
0807 19:44:50.890790 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:50.890796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:50.892052 (+ 1256us) server_negotiation.cc:407] Connection header received
0807 19:44:50.894326 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:50.894334 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:50.894498 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:50.894656 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:50.896702 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.897683 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.899493 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.900159 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.900318 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:50.904619 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:50.904626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:50.904629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:50.904668 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:50.907758 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:50.908684 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:50.908690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:50.908692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:50.908834 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:50.909354 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:50.909365 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:50.909367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:50.909671 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:50.909877 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:50.910692 (+ 815us) server_negotiation.cc:299] Negotiation successful
0807 19:44:50.910964 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":314,"threads_started":1}
I20260807 19:44:50.912609 9828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:50.890811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:59223)
0807 19:44:50.891833 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:50.891895 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:50.893956 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:50.894971 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:50.894981 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:50.895706 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:50.896472 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:50.896502 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.897879 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.897883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:50.899289 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:50.899303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:50.900434 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:50.900444 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:50.900746 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:50.901562 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:50.901644 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:50.904293 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:50.907950 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:50.907971 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:50.907988 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:50.908506 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:50.908971 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:50.908976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:50.908980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:50.909213 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:50.909845 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:50.909854 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:50.910260 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 19:44:50.910574 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 19:44:50.911238 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":384,"threads_started":1}
I20260807 19:44:50.914590 9825 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39291
I20260807 19:44:50.914933 9825 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:50.915861 9825 heartbeater.cc:511] Master 127.0.198.254:39291 requested a full tablet report, sending...
I20260807 19:44:50.918696 9627 ts_manager.cc:194] Registered new tserver with Master: 9265c58572674ed4a7bfd97882b54227 (127.0.198.193:43621)
I20260807 19:44:50.921125 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:59223
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:44:50.967463 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:51.336658 9831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:51.337018 9831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:51.337208 9831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:51.345923 9831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:51.346071 9831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:51.346277 9831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:51.357219 9831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:51.358665 9831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:51.360081 9831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:51.374697 9840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:51.374792 9837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:51.374817 9838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:51.375686 9831 server_base.cc:1034] running on GCE node
I20260807 19:44:51.376391 9831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:51.377513 9831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:51.378788 9831 hybrid_clock.cc:648] HybridClock initialized: now 1786131891378712 us; error 69 us; skew 500 ppm
I20260807 19:44:51.379197 9831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:51 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:51.382916 9831 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:44:51.384758 9831 webserver.cc:460] Webserver started at http://127.0.198.194:41761/ using document root <none> and password file <none>
I20260807 19:44:51.385506 9831 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:51.385627 9831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:51.385890 9831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:51.389129 9831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "00b304f8d7964ef0b051c11ea52d8089"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "6dc6c4232d81a7b77e8fa4cb4da6ecfa"
server_key_iv: "06e91540d8c5a2414a9b41de1edbde19"
server_key_version: "encryptionkey@0"
I20260807 19:44:51.390180 9831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "00b304f8d7964ef0b051c11ea52d8089"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "6dc6c4232d81a7b77e8fa4cb4da6ecfa"
server_key_iv: "06e91540d8c5a2414a9b41de1edbde19"
server_key_version: "encryptionkey@0"
I20260807 19:44:51.396303 9831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:44:51.400466 9847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:51.402300 9831 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:44:51.402534 9831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "00b304f8d7964ef0b051c11ea52d8089"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "6dc6c4232d81a7b77e8fa4cb4da6ecfa"
server_key_iv: "06e91540d8c5a2414a9b41de1edbde19"
server_key_version: "encryptionkey@0"
I20260807 19:44:51.402802 9831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:51.419438 9831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:51.425005 9831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:51.425431 9831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:51.426875 9831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:51.429023 9831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:51.429113 9831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:51.429284 9831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:51.429347 9831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:51.468436 9831 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35683
I20260807 19:44:51.468493 9967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35683 every 8 connection(s)
I20260807 19:44:51.469895 9831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:51.472247 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9831
I20260807 19:44:51.472446 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:51.473007 795 external_mini_cluster.cc:1428] Setting key 47ecee0907ab8d9d54a58ee1678cc6d0
Aug 07 19:44:51 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:51.495731 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:51.473078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:38385 (local address 127.0.198.254:39291)
0807 19:44:51.473635 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:51.473643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:51.474791 (+ 1148us) server_negotiation.cc:407] Connection header received
0807 19:44:51.477688 (+ 2897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:51.477694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:51.477789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:51.477894 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:51.480303 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:51.481171 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:51.483848 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:51.484333 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:51.484446 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:51.489150 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:51.489156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:51.489159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:51.489240 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:51.492212 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:51.493120 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:51.493125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:51.493128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:51.493271 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:51.493761 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:51.493770 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:51.493772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:51.494054 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:51.494222 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:51.495030 (+ 808us) server_negotiation.cc:299] Negotiation successful
0807 19:44:51.495289 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":261,"threads_started":1}
I20260807 19:44:51.496831 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:51.473790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:38385)
0807 19:44:51.474611 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:51.474666 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:51.477272 (+ 2606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:51.478213 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:51.478229 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:51.479114 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:51.480041 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:51.480071 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:51.481382 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:51.481391 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:51.483664 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:51.483692 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:51.484593 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:51.484603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:51.484964 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:51.485823 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:51.485862 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:51.488823 (+ 2961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:51.492419 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:51.492430 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:51.492457 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:51.492962 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:51.493401 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:51.493406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:51.493410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:51.493627 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:51.494192 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:51.494201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:51.494617 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 19:44:51.494911 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 19:44:51.495529 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":357,"threads_started":1}
I20260807 19:44:51.498565 9968 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39291
I20260807 19:44:51.498936 9968 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:51.499842 9968 heartbeater.cc:511] Master 127.0.198.254:39291 requested a full tablet report, sending...
I20260807 19:44:51.501693 9627 ts_manager.cc:194] Registered new tserver with Master: 00b304f8d7964ef0b051c11ea52d8089 (127.0.198.194:35683)
I20260807 19:44:51.502602 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:38385
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:44:51.548220 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:51.925451 9825 heartbeater.cc:503] Master 127.0.198.254:39291 was elected leader, sending a full tablet report...
W20260807 19:44:51.926327 9974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:51.926735 9974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:51.926932 9974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:51.936561 9974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:51.936764 9974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:51.936964 9974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:51.949282 9974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:51.951016 9974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:51.952641 9974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:51.970005 9979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:51.970278 9980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:51.973193 9982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:51.978015 9974 server_base.cc:1034] running on GCE node
I20260807 19:44:51.978905 9974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:51.980216 9974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:51.981570 9974 hybrid_clock.cc:648] HybridClock initialized: now 1786131891981455 us; error 35 us; skew 500 ppm
I20260807 19:44:51.981990 9974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:51 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:51.986274 9974 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:44:51.988317 9974 webserver.cc:460] Webserver started at http://127.0.198.195:44367/ using document root <none> and password file <none>
I20260807 19:44:51.989159 9974 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:51.989280 9974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:51.989627 9974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:44:51.993093 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "26f50b523e3a4b32b94fe6325e682880"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "1ad98643dcfaed5e3ea9f2bdc98a5633"
server_key_iv: "7e34f51c70e2a53ad75da0c254502210"
server_key_version: "encryptionkey@0"
I20260807 19:44:51.994032 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "26f50b523e3a4b32b94fe6325e682880"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "1ad98643dcfaed5e3ea9f2bdc98a5633"
server_key_iv: "7e34f51c70e2a53ad75da0c254502210"
server_key_version: "encryptionkey@0"
I20260807 19:44:52.000260 9974 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:44:52.004454 9990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:52.006093 9974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 19:44:52.006315 9974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "26f50b523e3a4b32b94fe6325e682880"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "1ad98643dcfaed5e3ea9f2bdc98a5633"
server_key_iv: "7e34f51c70e2a53ad75da0c254502210"
server_key_version: "encryptionkey@0"
I20260807 19:44:52.006536 9974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:52.030974 9974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:52.036696 9974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:52.037372 9974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:52.038687 9974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:52.040417 9974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:52.040546 9974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:52.040706 9974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:52.040781 9974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:52.081022 9974 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36575
I20260807 19:44:52.081069 10119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36575 every 8 connection(s)
I20260807 19:44:52.082486 9974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:52.083801 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 9974
I20260807 19:44:52.084023 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:52.084760 795 external_mini_cluster.cc:1428] Setting key 30f3ac69f6d0c7741483d897e3a07c19
Aug 07 19:44:52 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:44:52.117774 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:52.088459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:51851 (local address 127.0.198.254:39291)
0807 19:44:52.089489 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:52.089499 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:52.090511 (+ 1012us) server_negotiation.cc:407] Connection header received
0807 19:44:52.094365 (+ 3854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:52.094371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:52.094491 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:52.094679 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:52.097424 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.098566 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.102125 (+ 3559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.102593 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.102712 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:52.108116 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:52.108127 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:52.108135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:52.108241 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:52.113036 (+ 4795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.114392 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.114402 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.114409 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.114651 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.115516 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.115520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.115523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.115868 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:52.116012 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:52.117205 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0807 19:44:52.117444 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":724,"threads_started":1}
I20260807 19:44:52.119908 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:52.089153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:51851)
0807 19:44:52.090282 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:52.090347 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:52.093898 (+ 3551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:52.095047 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:52.095065 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:52.096110 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:52.097130 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:52.097172 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.098810 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.098820 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:52.101931 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:52.101954 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.102878 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.102889 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:52.103334 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:52.104288 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:52.104333 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:52.107747 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:52.113240 (+ 5493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:52.113253 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:52.113275 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:52.113936 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:52.114850 (+ 914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:52.114859 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:52.114867 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:52.115271 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:52.116037 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:52.116051 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:52.116683 (+ 632us) client_negotiation.cc:769] Sending connection context
0807 19:44:52.117082 (+ 399us) client_negotiation.cc:240] Negotiation successful
0807 19:44:52.117856 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":401,"threads_started":1}
I20260807 19:44:52.121835 10120 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39291
I20260807 19:44:52.122217 10120 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:52.123041 10120 heartbeater.cc:511] Master 127.0.198.254:39291 requested a full tablet report, sending...
I20260807 19:44:52.124806 9627 ts_manager.cc:194] Registered new tserver with Master: 26f50b523e3a4b32b94fe6325e682880 (127.0.198.195:36575)
I20260807 19:44:52.126003 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:51851
I20260807 19:44:52.128564 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
Entry for principal oryx/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab.
WARNING: no policy specified for oryx/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
Entry for principal oryx/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab.
WARNING: no policy specified for oryx/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
Entry for principal oryx/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab.
I20260807 19:44:52.343726 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9691
Aug 07 19:44:52 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:44:52.363253 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:52.345157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.0.198.193:43621)
0807 19:44:52.345933 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:52.345941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:52.345972 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:44:52.346055 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:52.346063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:52.346291 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:52.346459 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:52.348055 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.349233 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.350577 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.351210 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.351334 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:52.356319 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:52.356340 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:52.356357 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:52.356412 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:52.359991 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.360893 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.360909 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.360913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.361102 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.361543 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.361548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.361551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.362028 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:52.362322 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:52.362659 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 19:44:52.362913 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":389,"threads_started":1}
I20260807 19:44:52.364688 9820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:52.452325 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9691
I20260807 19:44:52.473065 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9831
Aug 07 19:44:52 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:44:52.492293 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:52.473995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.0.198.194:35683)
0807 19:44:52.474714 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:52.474721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:52.474749 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:44:52.474809 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:52.474815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:52.475012 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:52.475146 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:52.476880 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.478038 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.479463 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.480115 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.480232 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:52.485144 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:52.485162 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:52.485178 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:52.485241 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:52.488616 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.489544 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.489552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.489555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.489750 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.490155 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.490159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.490161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.490539 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:52.490821 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:52.491204 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 19:44:52.491935 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":409,"threads_started":1}
I20260807 19:44:52.493886 9963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:52.579561 9968 heartbeater.cc:503] Master 127.0.198.254:39291 was elected leader, sending a full tablet report...
I20260807 19:44:52.580778 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9831
W20260807 19:44:52.601007 9611 connection.cc:535] server connection from 127.0.198.194:38385 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:44:52.601420 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9974
Aug 07 19:44:52 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:44:52.620842 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:52.602417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.0.198.195:36575)
0807 19:44:52.602973 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:52.602982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:52.603014 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:44:52.603087 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:52.603094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:52.603346 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:52.603579 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:52.604917 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.606031 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.607529 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:52.608224 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:52.608344 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:52.613360 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:52.613374 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:52.613391 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:52.613438 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:52.616955 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.617941 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.617950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.617952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.618176 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:52.618699 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:52.618705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:52.618709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:52.619196 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:52.619558 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:52.620091 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 19:44:52.620383 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":268,"threads_started":1}
I20260807 19:44:52.622251 10114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:52.709673 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9974
I20260807 19:44:52.730464 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 9594
I20260807 19:44:52.732086 9659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:52.841984 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 9594
I20260807 19:44:52.870499 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39291
--webserver_interface=127.0.198.254
--webserver_port=39399
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:53.236884 10140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:53.237288 10140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:53.237464 10140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:53.245307 10140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:53.245442 10140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:53.245489 10140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:53.245540 10140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:53.245585 10140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:53.256739 10140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=39399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:53.258159 10140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:53.259588 10140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:53.272591 10149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:53.272708 10145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:53.272655 10146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:53.273471 10140 server_base.cc:1034] running on GCE node
I20260807 19:44:53.274403 10140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:53.276217 10140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:53.277520 10140 hybrid_clock.cc:648] HybridClock initialized: now 1786131893277471 us; error 55 us; skew 500 ppm
I20260807 19:44:53.277961 10140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:53 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131893, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:53.281770 10140 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:44:53.283675 10140 webserver.cc:460] Webserver started at http://127.0.198.254:39399/ using document root <none> and password file <none>
I20260807 19:44:53.284441 10140 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:53.284574 10140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:53.290810 10140 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 19:44:53.294374 10156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:53.296309 10140 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:44:53.296532 10140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "e4160594768a4b9fb9f32bf754e4dd48"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "ae8cac50385d4fdc896552df2bcf26cb"
server_key_iv: "8d5e86eef98c0a238d20efc117483fd3"
server_key_version: "encryptionkey@0"
I20260807 19:44:53.298470 10140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:53.329257 10140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:53.334287 10140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:53.334616 10140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:53.355985 10140 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:39291
I20260807 19:44:53.356006 10213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:39291 every 8 connection(s)
I20260807 19:44:53.357648 10140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:53.364421 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10140
I20260807 19:44:53.365540 10214 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:44:53.365561 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:43621
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:39291
--webserver_port=39917
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:53.371289 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Bootstrap starting.
I20260807 19:44:53.393885 10214 log.cc:824] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:53.404538 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:44:53.405186 10214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Bootstrap complete.
I20260807 19:44:53.415426 10214 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:53.416729 10214 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4160594768a4b9fb9f32bf754e4dd48, State: Initialized, Role: FOLLOWER
I20260807 19:44:53.417598 10214 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:53.417887 10214 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:53.418020 10214 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:53.418237 10214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:44:53.422777 10214 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:53.423441 10214 leader_election.cc:302] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4160594768a4b9fb9f32bf754e4dd48; no voters:
I20260807 19:44:53.424228 10214 leader_election.cc:288] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:44:53.424401 10218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:44:53.425838 10218 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [term 2 LEADER]: Becoming Leader. State: Replica: e4160594768a4b9fb9f32bf754e4dd48, State: Running, Role: LEADER
I20260807 19:44:53.426607 10218 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } }
I20260807 19:44:53.426959 10214 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:53.429852 10220 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4160594768a4b9fb9f32bf754e4dd48. Latest consensus state: current_term: 2 leader_uuid: "e4160594768a4b9fb9f32bf754e4dd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } } }
I20260807 19:44:53.430552 10220 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:53.430449 10219 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e4160594768a4b9fb9f32bf754e4dd48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4160594768a4b9fb9f32bf754e4dd48" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39291 } } }
I20260807 19:44:53.432627 10219 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:53.433480 10228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:53.451771 10228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:53.453650 10228 catalog_manager.cc:1259] Loaded cluster ID: 96c412f2019c4c72ab9afe6f10382770
I20260807 19:44:53.453884 10228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:53.458600 10228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:53.460750 10228 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e4160594768a4b9fb9f32bf754e4dd48: Loaded TSK: 0
I20260807 19:44:53.462827 10228 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:44:53.755159 10216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:53.755620 10216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:53.755803 10216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:53.763629 10216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:53.763805 10216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:53.763986 10216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:44:53.774665 10216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.193.keytab
--principal=oryx/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:43621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=39917
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:53.776286 10216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:53.777735 10216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:53.793061 10243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:53.793071 10244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:53.795730 10246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:53.798058 10216 server_base.cc:1034] running on GCE node
I20260807 19:44:53.798906 10216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:53.800176 10216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:53.801417 10216 hybrid_clock.cc:648] HybridClock initialized: now 1786131893801348 us; error 61 us; skew 500 ppm
I20260807 19:44:53.801783 10216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:53 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131893, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:53.805450 10216 init.cc:377] Logged in from keytab as oryx/127.0.198.193@KRBTEST.COM (short username oryx)
I20260807 19:44:53.807489 10216 webserver.cc:460] Webserver started at http://127.0.198.193:39917/ using document root <none> and password file <none>
I20260807 19:44:53.808389 10216 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:53.808573 10216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:53.814715 10216 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 19:44:53.818482 10253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:53.820624 10216 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:53.820894 10216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "9265c58572674ed4a7bfd97882b54227"
format_stamp: "Formatted at 2026-08-07 19:44:50 on dist-test-slave-btw8"
server_key: "34de1cfa91ad6938b36e49a99cac2014"
server_key_iv: "4fa78888c193774f5cde9d5e320957ee"
server_key_version: "encryptionkey@0"
I20260807 19:44:53.822120 10216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:53.842587 10216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:53.847684 10216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:53.848006 10216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:53.849278 10216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:53.850802 10216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:53.850900 10216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:53.851042 10216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:53.851145 10216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:53.887570 10216 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:43621
I20260807 19:44:53.887663 10373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:43621 every 8 connection(s)
I20260807 19:44:53.889200 10216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:53.892570 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10216
I20260807 19:44:53.893244 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:35683
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:39291
--webserver_port=41761
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:53 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:53 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:44:54.275067 10379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:54.275473 10379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:54.275643 10379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:54.283305 10379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:54.283558 10379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:54.283747 10379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:44:54.294123 10379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.194.keytab
--principal=oryx/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:35683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=41761
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:54.295603 10379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:54.296967 10379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:54.317186 10386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:54.317994 10385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:54.317340 10388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:54.319588 10379 server_base.cc:1034] running on GCE node
I20260807 19:44:54.320400 10379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:54.321587 10379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:54.325018 10379 hybrid_clock.cc:648] HybridClock initialized: now 1786131894324895 us; error 105 us; skew 500 ppm
I20260807 19:44:54.325496 10379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:54 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131894, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:54.330067 10379 init.cc:377] Logged in from keytab as oryx/127.0.198.194@KRBTEST.COM (short username oryx)
I20260807 19:44:54.332190 10379 webserver.cc:460] Webserver started at http://127.0.198.194:41761/ using document root <none> and password file <none>
I20260807 19:44:54.333068 10379 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:54.333258 10379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:54.340343 10379 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260807 19:44:54.344281 10395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:54.346248 10379 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:44:54.346486 10379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "00b304f8d7964ef0b051c11ea52d8089"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "6dc6c4232d81a7b77e8fa4cb4da6ecfa"
server_key_iv: "06e91540d8c5a2414a9b41de1edbde19"
server_key_version: "encryptionkey@0"
I20260807 19:44:54.347656 10379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:54.375607 10379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:54.380573 10379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:54.380895 10379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:54.382055 10379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:54.383728 10379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:54.383870 10379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:54.384032 10379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:54.384141 10379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:54.437157 10379 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35683
I20260807 19:44:54.437230 10518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35683 every 8 connection(s)
I20260807 19:44:54.438944 10379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:54.446349 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10379
I20260807 19:44:54.447178 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:36575
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:39291
--webserver_port=44367
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:54 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:54 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:44:55.159334 10523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:55.159720 10523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:55.159885 10523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:55.168244 10523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:55.168435 10523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:55.168653 10523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:44:55.179217 10523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/krb5kdc/oryx_127.0.198.195.keytab
--principal=oryx/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:36575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=44367
--tserver_master_addrs=127.0.198.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:44:55.180787 10523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:55.182191 10523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:55.202109 10530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:55.203069 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:55.203698 10529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:55.203892 10523 server_base.cc:1034] running on GCE node
I20260807 19:44:55.204756 10523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:55.205996 10523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:55.209391 10523 hybrid_clock.cc:648] HybridClock initialized: now 1786131895209298 us; error 93 us; skew 500 ppm
I20260807 19:44:55.209810 10523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131895, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:55.213867 10523 init.cc:377] Logged in from keytab as oryx/127.0.198.195@KRBTEST.COM (short username oryx)
I20260807 19:44:55.215915 10523 webserver.cc:460] Webserver started at http://127.0.198.195:44367/ using document root <none> and password file <none>
I20260807 19:44:55.216747 10523 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:55.216874 10523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:55.223094 10523 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 19:44:55.226814 10539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:55.228987 10523 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:55.229234 10523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "26f50b523e3a4b32b94fe6325e682880"
format_stamp: "Formatted at 2026-08-07 19:44:51 on dist-test-slave-btw8"
server_key: "1ad98643dcfaed5e3ea9f2bdc98a5633"
server_key_iv: "7e34f51c70e2a53ad75da0c254502210"
server_key_version: "encryptionkey@0"
I20260807 19:44:55.230350 10523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:55.250170 10523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:55.256465 10523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:55.256817 10523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:55.258195 10523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:55.259944 10523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:55.260038 10523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:55.260118 10523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:55.260165 10523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:55.310768 10523 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36575
I20260807 19:44:55.311069 10659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36575 every 8 connection(s)
I20260807 19:44:55.313083 10523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:55.316072 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10523
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:44:55.373147 10376 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.464s user 0.012s sys 0.007s
I20260807 19:44:55.375699 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:53.893059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:44653)
0807 19:44:53.894128 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:53.894188 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:53.897094 (+ 2906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:53.898762 (+ 1668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:53.898777 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:53.899691 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:53.900491 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:53.900522 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:53.903278 (+ 2756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:53.903286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:53.905731 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:53.905754 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:53.907532 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:53.907552 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:53.907862 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:53.908841 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:53.908882 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.374424 (+1465542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":969,"threads_started":1}
W20260807 19:44:55.375705 10377 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:53.893842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:44653 (local address 127.0.198.254:39291)
0807 19:44:53.894589 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:53.894606 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:53.894664 (+ 58us) server_negotiation.cc:407] Connection header received
0807 19:44:53.897751 (+ 3087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:53.897765 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:53.898193 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:53.898530 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:53.900808 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:53.902997 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:53.905973 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:53.907135 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:53.907371 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.374814 (+1467443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:44653: BlockingRecv error: recv got EOF from 127.0.198.193:44653 (error 108)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":1103,"threads_started":1}
W20260807 19:44:55.377038 10374 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39291 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.198.254:39291: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
I20260807 19:44:55.379005 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.317224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.0.198.254:39291)
0807 19:44:55.335175 (+ 17951us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.335182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.336801 (+ 1619us) server_negotiation.cc:407] Connection header received
0807 19:44:55.338286 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.338294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.338385 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.338522 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.342554 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.343985 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.348003 (+ 4018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.349067 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.349256 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.364044 (+ 14788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:55.364065 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:55.364107 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:55.364211 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:55.370971 (+ 6760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:55.372730 (+ 1759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:55.372745 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:55.372764 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:55.373144 (+ 380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:55.373890 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:55.373899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:55.373906 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:55.375005 (+ 1099us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:55.377256 (+ 2251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:55.377724 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 19:44:55.378340 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17763,"thread_start_us":19789,"threads_started":1}
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:55.396441 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.377808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:53771)
0807 19:44:55.378189 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:55.378204 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:55.378395 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:55.379137 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:55.379145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:55.379780 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:55.380116 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.380134 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.383312 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.383324 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.386488 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.386522 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.388128 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.388139 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.388462 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.389405 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:55.389445 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.395893 (+ 6448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283}
W20260807 19:44:55.399997 10377 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:55.377869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:53771 (local address 127.0.198.254:39291)
0807 19:44:55.378068 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.378072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.378292 (+ 220us) server_negotiation.cc:407] Connection header received
0807 19:44:55.378529 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.378539 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.378662 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.378815 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.380391 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.382842 (+ 2451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.386780 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.387815 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.388022 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.399622 (+ 11600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:53771: BlockingRecv error: recv got EOF from 127.0.198.193:53771 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:55.419041 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.397823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:58243)
0807 19:44:55.400652 (+ 2829us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:55.400681 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:55.400870 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:55.401605 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:55.401613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:55.402722 (+ 1109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:55.403066 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.403086 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.405309 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.405318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.407876 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.407908 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.410201 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.410213 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.410552 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.411629 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:55.411668 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.418507 (+ 6839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2658,"thread_start_us":4542,"threads_started":1}
W20260807 19:44:55.419521 10664 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:55.398615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:58243 (local address 127.0.198.254:39291)
0807 19:44:55.398929 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.398946 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.400777 (+ 1831us) server_negotiation.cc:407] Connection header received
0807 19:44:55.401036 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.401044 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.401197 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.401361 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.403300 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.405022 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.408195 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.409314 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.409558 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.419014 (+ 9456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:58243: BlockingRecv error: recv got EOF from 127.0.198.193:58243 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:44:55.419881 10374 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:44:55.625469 10522 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.166s user 0.006s sys 0.008s
W20260807 19:44:55.629302 10521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:54.443279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:51637 (local address 127.0.198.254:39291)
0807 19:44:54.443888 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:54.443893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:54.445634 (+ 1741us) server_negotiation.cc:407] Connection header received
0807 19:44:54.448989 (+ 3355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:54.449000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:54.449190 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:54.449379 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:54.452018 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:54.453855 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:54.456685 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:54.457723 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:54.457921 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.628808 (+1170887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:51637: BlockingRecv error: recv got EOF from 127.0.198.194:51637 (error 108)
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":352,"threads_started":1}
I20260807 19:44:55.629343 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:54.444411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:51637)
0807 19:44:54.445429 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:54.445488 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:54.448487 (+ 2999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:54.449694 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:54.449710 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:54.450611 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:54.451701 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:54.451736 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:54.454105 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:54.454114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:54.456464 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:54.456485 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:54.458058 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:54.458068 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:54.458452 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:54.459575 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:54.459600 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.627032 (+1167432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":415,"threads_started":1}
W20260807 19:44:55.630517 10519 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39291 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.198.254:39291: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:55.642446 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.631318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:44471)
0807 19:44:55.631555 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:55.631578 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:55.631716 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:55.632101 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:55.632106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:55.632545 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:55.632778 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.632788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.633911 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.633916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.635016 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.635029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.635978 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.635984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.636165 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.637059 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:55.637089 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.642089 (+ 5000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 19:44:55.642663 10521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:55.631467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:44471 (local address 127.0.198.254:39291)
0807 19:44:55.631662 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.631669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.631689 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:44:55.631870 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.631875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.631957 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.632077 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.632930 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.633740 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.635199 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.635793 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.635941 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.642370 (+ 6429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:44471: BlockingRecv error: recv got EOF from 127.0.198.194:44471 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:55 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:55.657599 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.643660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:33517)
0807 19:44:55.643933 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:55.643949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:55.644235 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:55.644617 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:55.644621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:55.645040 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:55.645303 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.645317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.646581 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.646590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.649227 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.649262 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.650280 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.650290 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.650622 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.651661 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:55.651721 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:55.657267 (+ 5546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163}
W20260807 19:44:55.658039 10519 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:44:55.658234 10521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:55.644002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:33517 (local address 127.0.198.254:39291)
0807 19:44:55.644277 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.644285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.644308 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:55.644375 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.644381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.644462 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.644574 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.645443 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.646367 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.649514 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.650056 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.650182 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.657550 (+ 7368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:33517: BlockingRecv error: recv got EOF from 127.0.198.194:33517 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:56.438737 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:56.420860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:59933)
0807 19:44:56.421492 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:56.421521 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:56.421711 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:56.422277 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:56.422284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:56.422792 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:56.422968 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.422978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.424278 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.424292 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.425374 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.425389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.427179 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.427186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.427458 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.434197 (+ 6739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:56.434223 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:56.438239 (+ 4016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":373,"threads_started":1}
W20260807 19:44:56.439229 10669 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:56.421041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:59933 (local address 127.0.198.254:39291)
0807 19:44:56.421918 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:56.421923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:56.421942 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:44:56.421997 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:56.422001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:56.422085 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:56.422230 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:56.423183 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.424093 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.425681 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.426861 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.427103 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.438785 (+ 11682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:59933: BlockingRecv error: recv got EOF from 127.0.198.193:59933 (error 108)
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":515,"threads_started":1}
W20260807 19:44:56.555473 10665 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.209s user 0.007s sys 0.014s
W20260807 19:44:56.556834 10663 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:55.317552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:60735 (local address 127.0.198.254:39291)
0807 19:44:55.337379 (+ 19827us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:55.337389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:55.337414 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:44:55.337505 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:55.337513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:55.337635 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:55.337837 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:55.340802 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.341912 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.344672 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.345129 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.345303 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.556571 (+1211268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:60735: BlockingRecv error: recv got EOF from 127.0.198.195:60735 (error 108)
Metrics: {"server-negotiator.queue_time_us":19632,"thread_start_us":17199,"threads_started":1}
I20260807 19:44:56.557552 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:55.318118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:60735)
0807 19:44:55.329753 (+ 11635us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:55.329838 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:55.332672 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:55.338297 (+ 5625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:55.338396 (+ 99us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:55.339537 (+ 1141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:55.340549 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.340573 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.342098 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.342106 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.344452 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:55.344476 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:55.345420 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:55.345431 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:55.345744 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:55.346759 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:55.346796 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:56.556306 (+1209510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":11214,"thread_start_us":10894,"threads_started":1}
W20260807 19:44:56.558770 10660 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:39291 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.198.254:39291: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:56.571975 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:56.559787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:53459)
0807 19:44:56.560168 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:56.560196 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:56.560363 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:56.560887 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:56.560892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:56.561356 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:56.561563 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.561573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.562956 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.562962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.564624 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.564646 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.565466 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.565472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.565610 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.566446 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:56.566473 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:56.571708 (+ 5235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126}
W20260807 19:44:56.572309 10663 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:56.559861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:53459 (local address 127.0.198.254:39291)
0807 19:44:56.560224 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:56.560233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:56.560266 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:44:56.560616 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:56.560621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:56.560717 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:56.560849 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:56.561736 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.562729 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.564789 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.565241 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.565352 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.572004 (+ 6652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:53459: BlockingRecv error: recv got EOF from 127.0.198.195:53459 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:56.582243 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:56.572860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:50819)
0807 19:44:56.573055 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:56.573072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:56.573203 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:56.573543 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:56.573546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:56.573866 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:56.574034 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.574042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.575128 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.575133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.576463 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.576475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.577275 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.577284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.577432 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.578040 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:56.578065 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:56.582027 (+ 3962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:44:56.582624 10660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:44:56.582720 10663 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:56.572954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:50819 (local address 127.0.198.254:39291)
0807 19:44:56.573124 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:56.573129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:56.573147 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:44:56.573314 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:56.573317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:56.573407 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:56.573491 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:56.574156 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.574949 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.576631 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.577096 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.577212 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.582384 (+ 5172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:50819: BlockingRecv error: recv got EOF from 127.0.198.195:50819 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:56 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:56.669890 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:56.658903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:39977)
0807 19:44:56.659557 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:56.659579 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:56.659718 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:56.660205 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:56.660209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:56.660688 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:56.660841 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.660850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.662078 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.662084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.663270 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:56.663286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.664279 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.664286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:56.664517 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.665254 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:56.665273 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:56.669623 (+ 4350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":372,"threads_started":1}
W20260807 19:44:56.670073 10663 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:56.659146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:39977 (local address 127.0.198.254:39291)
0807 19:44:56.659454 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:56.659461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:56.659680 (+ 219us) server_negotiation.cc:407] Connection header received
0807 19:44:56.659904 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:56.659908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:56.660009 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:56.660128 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:56.661035 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.661881 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.663565 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:56.664051 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:56.664242 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:56.669850 (+ 5608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:39977: BlockingRecv error: recv got EOF from 127.0.198.194:39977 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:57.451630 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:57.440496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:54203)
0807 19:44:57.441164 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:57.441210 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:57.441500 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:57.442005 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:57.442011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:57.442583 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:57.442778 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.442789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.444070 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.444076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.445029 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.445042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.445846 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.445852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.446064 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.446790 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:57.446810 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:57.451219 (+ 4409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":343,"threads_started":1}
W20260807 19:44:57.451934 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:57.440538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:54203 (local address 127.0.198.254:39291)
0807 19:44:57.441532 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:57.441539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:57.441566 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:44:57.441694 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:57.441700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:57.441794 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:57.441923 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:57.442988 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.443889 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.445215 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.445660 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.445812 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.451584 (+ 5772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:54203: BlockingRecv error: recv got EOF from 127.0.198.193:54203 (error 108)
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":753,"threads_started":1}
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:57.593665 10673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:57.583329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:57223)
0807 19:44:57.583963 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:57.583981 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:57.584117 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:57.584578 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:57.584582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:57.585066 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:57.585222 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.585231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.586434 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.586439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.587453 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.587467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.588280 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.588285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.588422 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.589209 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:57.589228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:57.593350 (+ 4122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":341,"threads_started":1}
W20260807 19:44:57.593863 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:57.583505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:57223 (local address 127.0.198.254:39291)
0807 19:44:57.583762 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:57.583773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:57.584084 (+ 311us) server_negotiation.cc:407] Connection header received
0807 19:44:57.584292 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:57.584297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:57.584396 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:57.584511 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:57.585412 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.586258 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.587617 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.588072 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.588192 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.593597 (+ 5405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:57223: BlockingRecv error: recv got EOF from 127.0.198.195:57223 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:57 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:57.681434 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:57.671334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:40527)
0807 19:44:57.671923 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:57.671948 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:57.672145 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:57.672511 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:57.672514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:57.673007 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:57.673202 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.673211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.674299 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.674303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.675257 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:57.675269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.676088 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.676093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:57.676225 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.676879 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:57.676897 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:57.681065 (+ 4168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":310,"threads_started":1}
W20260807 19:44:57.681602 10672 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:57.671449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:40527 (local address 127.0.198.254:39291)
0807 19:44:57.671627 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:57.671632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:57.672093 (+ 461us) server_negotiation.cc:407] Connection header received
0807 19:44:57.672297 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:57.672301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:57.672369 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:57.672456 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:57.673347 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.674132 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.675463 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.675903 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.676021 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.681311 (+ 5290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:40527: BlockingRecv error: recv got EOF from 127.0.198.194:40527 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:58.463164 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:58.452766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:60689)
0807 19:44:58.453373 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:58.453403 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:58.453525 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:58.453904 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:58.453908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:58.454296 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:58.454492 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.454501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.455681 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.455685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.456642 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.456655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.457496 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.457501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.457704 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.458401 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:58.458419 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:58.462804 (+ 4385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":352,"threads_started":1}
W20260807 19:44:58.463402 10676 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:58.452967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:60689 (local address 127.0.198.254:39291)
0807 19:44:58.453510 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:58.453515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:58.453535 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:44:58.453654 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:58.453658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:58.453728 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:58.453826 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:58.454725 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.455524 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.456852 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.457309 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.457421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.463062 (+ 5641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:60689: BlockingRecv error: recv got EOF from 127.0.198.193:60689 (error 108)
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":291,"threads_started":1}
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:58.605674 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:58.594822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:44103)
0807 19:44:58.595784 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:58.595810 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:58.595977 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:58.596595 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:58.596600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:58.597042 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:58.597227 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.597236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.598301 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.598306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.599464 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.599477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.600240 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.600245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.600428 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.601065 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:58.601083 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:58.605306 (+ 4223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":357,"threads_started":1}
W20260807 19:44:58.605818 10676 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:58.594843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:44103 (local address 127.0.198.254:39291)
0807 19:44:58.595028 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:58.595036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:58.595891 (+ 855us) server_negotiation.cc:407] Connection header received
0807 19:44:58.596343 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:58.596349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:58.596440 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:58.596565 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:58.597370 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.598152 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.599603 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.600078 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.600229 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.605519 (+ 5290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:44103: BlockingRecv error: recv got EOF from 127.0.198.195:44103 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:58 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:58.695916 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:58.682934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:41933)
0807 19:44:58.683728 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:58.683762 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:58.683952 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:58.684404 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:58.684413 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:58.685045 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:58.685361 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.685378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.686794 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.686804 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.688950 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.688976 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.689820 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.689830 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.690091 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.690829 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:58.690862 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:58.695556 (+ 4694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":425,"threads_started":1}
W20260807 19:44:58.696112 10676 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:58.683017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41933 (local address 127.0.198.254:39291)
0807 19:44:58.683331 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:58.683341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:58.683896 (+ 555us) server_negotiation.cc:407] Connection header received
0807 19:44:58.684106 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:58.684110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:58.684203 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:58.684294 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:58.685531 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.686663 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.689167 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.689636 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.689753 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.695800 (+ 6047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:41933: BlockingRecv error: recv got EOF from 127.0.198.194:41933 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:59.475123 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:59.464330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:59235)
0807 19:44:59.464947 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:59.464970 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:59.465119 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:59.465422 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:59.465426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:59.465912 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:59.466085 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.466095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.467124 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.467128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.468475 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.468490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.469506 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.469511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.469749 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.470571 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:59.470600 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:59.474859 (+ 4259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":310,"threads_started":1}
W20260807 19:44:59.475468 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:59.464586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:59235 (local address 127.0.198.254:39291)
0807 19:44:59.465125 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:59.465130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:59.465149 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:44:59.465204 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:59.465207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:59.465270 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:59.465367 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:59.466223 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.466957 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.468690 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.469295 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.469464 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.475126 (+ 5662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:59235: BlockingRecv error: recv got EOF from 127.0.198.193:59235 (error 108)
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":265,"threads_started":1}
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:59.618271 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:59.607090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:52633)
0807 19:44:59.607819 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:59.607837 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:59.608023 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:59.608413 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:59.608418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:59.608843 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:59.609029 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.609038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.610206 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.610215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.611514 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.611529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.612357 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.612364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.612565 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.613217 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:59.613236 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:59.617978 (+ 4742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":380,"threads_started":1}
W20260807 19:44:59.618629 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:59.607339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:52633 (local address 127.0.198.254:39291)
0807 19:44:59.607622 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:59.607629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:59.607939 (+ 310us) server_negotiation.cc:407] Connection header received
0807 19:44:59.608182 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:59.608186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:59.608256 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:59.608376 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:59.609205 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.610037 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.611702 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.612176 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.612341 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.618314 (+ 5973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:52633: BlockingRecv error: recv got EOF from 127.0.198.195:52633 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:44:59 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:44:59.708318 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:59.697234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:34525)
0807 19:44:59.697827 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:59.697849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:59.697973 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:59.698363 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:59.698367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:59.698808 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:59.698998 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.699007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.700724 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.700730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.702177 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.702193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.702967 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.702972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.703117 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.703907 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:59.703928 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:59.708039 (+ 4111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":333,"threads_started":1}
W20260807 19:44:59.708571 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:44:59.697339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:34525 (local address 127.0.198.254:39291)
0807 19:44:59.697554 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:59.697559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:59.697951 (+ 392us) server_negotiation.cc:407] Connection header received
0807 19:44:59.698092 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:59.698095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:59.698182 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:59.698291 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:59.699295 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.700396 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.702322 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.702759 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.702906 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.708288 (+ 5382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:34525: BlockingRecv error: recv got EOF from 127.0.198.194:34525 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:00.485735 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.476299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:60357)
0807 19:45:00.476852 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:00.476874 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:00.477007 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:00.477290 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:00.477293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:00.477728 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:00.477880 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.477888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.478971 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.478975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.479989 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.480002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.480820 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.480824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.480950 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.481664 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:00.481685 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:00.485492 (+ 3807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":242,"threads_started":1}
W20260807 19:45:00.486184 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:00.476437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:60357 (local address 127.0.198.254:39291)
0807 19:45:00.477013 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.477018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.477035 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:45:00.477087 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.477091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.477152 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.477247 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.478025 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.478836 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.480173 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.480678 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.480791 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.485888 (+ 5097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:60357: BlockingRecv error: recv got EOF from 127.0.198.193:60357 (error 108)
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":351,"threads_started":1}
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:00.631217 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.619422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:52863)
0807 19:45:00.620115 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:00.620134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:00.620318 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:00.620717 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:00.620722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:00.621211 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:00.621407 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.621417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.622605 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.622611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.623845 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.623860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.624692 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.624699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.624909 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.625719 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:00.625743 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:00.630825 (+ 5082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":393,"threads_started":1}
W20260807 19:45:00.631417 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:00.619593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:52863 (local address 127.0.198.254:39291)
0807 19:45:00.619866 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.619876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.620237 (+ 361us) server_negotiation.cc:407] Connection header received
0807 19:45:00.620461 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.620466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.620543 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.620663 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.621578 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.622431 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.624019 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.624506 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.624667 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.631086 (+ 6419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:52863: BlockingRecv error: recv got EOF from 127.0.198.195:52863 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:00 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:00.719183 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.709356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:44497)
0807 19:45:00.710073 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:00.710095 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:00.710221 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:00.710548 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:00.710552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:00.710974 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:00.711166 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.711176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.712331 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.712335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.713293 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.713306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.714067 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.714071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.714197 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.714941 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:00.714960 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:00.718831 (+ 3871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":447,"threads_started":1}
W20260807 19:45:00.719650 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:00.709545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:44497 (local address 127.0.198.254:39291)
0807 19:45:00.709721 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.709726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.710175 (+ 449us) server_negotiation.cc:407] Connection header received
0807 19:45:00.710327 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.710331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.710408 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.710492 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.711317 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.712177 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.713451 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.713917 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.714024 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.719082 (+ 5058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:44497: BlockingRecv error: recv got EOF from 127.0.198.194:44497 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:01.501894 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:01.487016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:60867)
0807 19:45:01.487653 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:01.487677 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:01.487820 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:01.488186 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:01.488203 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:01.488662 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:01.488829 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.488838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.489966 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.489971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.490957 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.490969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.491861 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.491867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.492062 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.497151 (+ 5089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:01.497182 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:01.501511 (+ 4329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":291,"threads_started":1}
W20260807 19:45:01.502110 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:01.487099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:60867 (local address 127.0.198.254:39291)
0807 19:45:01.487822 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:01.487828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:01.487848 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:45:01.487960 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:01.487964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:01.488029 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:01.488146 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:01.488998 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.489805 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.491132 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.491668 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.491815 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.501805 (+ 9990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:60867: BlockingRecv error: recv got EOF from 127.0.198.193:60867 (error 108)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":430,"threads_started":1}
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:01.643106 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:01.632496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:60271)
0807 19:45:01.633223 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:01.633241 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:01.633370 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:01.633711 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:01.633714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:01.634340 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:01.634498 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.634508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.635848 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.635853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.636958 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.636974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.638066 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.638071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.638208 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.638964 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:01.638983 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:01.642862 (+ 3879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":445,"threads_started":1}
W20260807 19:45:01.643467 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:01.632627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:60271 (local address 127.0.198.254:39291)
0807 19:45:01.632817 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:01.632822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:01.633331 (+ 509us) server_negotiation.cc:407] Connection header received
0807 19:45:01.633485 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:01.633489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:01.633558 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:01.633655 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:01.634652 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.635667 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.637191 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.637869 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.638011 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.643166 (+ 5155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:60271: BlockingRecv error: recv got EOF from 127.0.198.195:60271 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:01 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:01.731287 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:01.720337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:42745)
0807 19:45:01.720840 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:01.720864 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:01.720995 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:01.721373 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:01.721376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:01.721796 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:01.721976 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.721985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.723241 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.723245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.724336 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.724353 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.725244 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.725250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.725392 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.726285 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:01.726315 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:01.730893 (+ 4578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
W20260807 19:45:01.731446 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:01.720532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:42745 (local address 127.0.198.254:39291)
0807 19:45:01.720806 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:01.720814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:01.720978 (+ 164us) server_negotiation.cc:407] Connection header received
0807 19:45:01.721121 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:01.721126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:01.721221 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:01.721326 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:01.722157 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.723019 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.724532 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.725015 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.725155 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.731118 (+ 5963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:42745: BlockingRecv error: recv got EOF from 127.0.198.194:42745 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:02.514038 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:02.503021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:49635)
0807 19:45:02.503715 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:02.503732 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:02.503851 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:02.504244 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:02.504248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:02.504763 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:02.504955 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.504964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.506064 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.506069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.507049 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.507061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.508181 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.508185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.508310 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.509031 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:02.509054 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:02.513632 (+ 4578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":314,"threads_started":1}
W20260807 19:45:02.514189 10692 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:02.503139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:49635 (local address 127.0.198.254:39291)
0807 19:45:02.503712 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:02.503717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:02.503822 (+ 105us) server_negotiation.cc:407] Connection header received
0807 19:45:02.503982 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:02.503986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:02.504059 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:02.504185 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:02.505113 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.505878 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.507225 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.508005 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.508131 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.513854 (+ 5723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:49635: BlockingRecv error: recv got EOF from 127.0.198.193:49635 (error 108)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":356,"threads_started":1}
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:02.654309 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:02.644264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:57881)
0807 19:45:02.644873 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:02.644889 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:02.645063 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:02.645437 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:02.645442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:02.645906 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:02.646069 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.646077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.647182 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.647188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.648304 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.648318 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.649123 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.649129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.649318 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.649948 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:02.649966 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:02.654033 (+ 4067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":313,"threads_started":1}
W20260807 19:45:02.654582 10692 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:02.644416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:57881 (local address 127.0.198.254:39291)
0807 19:45:02.644671 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:02.644678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:02.644990 (+ 312us) server_negotiation.cc:407] Connection header received
0807 19:45:02.645211 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:02.645221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:02.645287 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:02.645397 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:02.646250 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.647010 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.648496 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.648941 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.649083 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.654307 (+ 5224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:57881: BlockingRecv error: recv got EOF from 127.0.198.195:57881 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:02 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:02.745311 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:02.732312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:40725)
0807 19:45:02.733000 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:02.733030 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:02.733197 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:02.733575 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:02.733581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:02.734203 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:02.734514 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.734529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.735951 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.735960 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.737801 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.737822 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.738851 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.738866 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.739117 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.739954 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:02.739983 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:02.744917 (+ 4934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":297,"threads_started":1}
W20260807 19:45:02.745828 10692 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:02.732352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:40725 (local address 127.0.198.254:39291)
0807 19:45:02.732581 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:02.732588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:02.733130 (+ 542us) server_negotiation.cc:407] Connection header received
0807 19:45:02.733328 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:02.733332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:02.733410 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:02.733496 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:02.734702 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.735741 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.738025 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.738467 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.738590 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.745203 (+ 6613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:40725: BlockingRecv error: recv got EOF from 127.0.198.194:40725 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:03.526803 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.515309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:42969)
0807 19:45:03.516055 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.516088 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.516232 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.516824 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.516829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.517291 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:03.517495 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.517505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.518729 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.518734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.519776 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.519791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.520758 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.520766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.520945 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.521853 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:03.521874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:03.526371 (+ 4497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":327,"threads_started":1}
W20260807 19:45:03.527138 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:03.515574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:42969 (local address 127.0.198.254:39291)
0807 19:45:03.516479 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.516483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.516501 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:45:03.516557 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.516561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.516623 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.516725 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.517715 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.518553 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.519976 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.520435 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.520624 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.526790 (+ 6166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:42969: BlockingRecv error: recv got EOF from 127.0.198.193:42969 (error 108)
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":669,"threads_started":1}
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:03.667038 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.655554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:43739)
0807 19:45:03.656400 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.656418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.656553 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.656921 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.656925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.657392 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:03.657579 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.657588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.658873 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.658877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.660219 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.660233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.661422 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.661467 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.661772 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.662647 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:03.662671 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:03.666776 (+ 4105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":443,"threads_started":1}
W20260807 19:45:03.667429 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:03.655816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:43739 (local address 127.0.198.254:39291)
0807 19:45:03.656022 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.656027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.656504 (+ 477us) server_negotiation.cc:407] Connection header received
0807 19:45:03.656663 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.656667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.656735 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.656860 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.657825 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.658706 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.660432 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.661189 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.661377 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.667046 (+ 5669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:43739: BlockingRecv error: recv got EOF from 127.0.198.195:43739 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:03 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:03.758095 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.746663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:43875)
0807 19:45:03.747334 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.747380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.747520 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.747949 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.747953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.748371 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:03.748551 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.748560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.750209 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.750214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.751680 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.751698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.752841 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.752849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.753028 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.753721 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:03.753740 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:03.757763 (+ 4023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":401,"threads_started":1}
W20260807 19:45:03.758278 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:03.746788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:43875 (local address 127.0.198.254:39291)
0807 19:45:03.747025 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.747030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.747471 (+ 441us) server_negotiation.cc:407] Connection header received
0807 19:45:03.747693 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.747699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.747799 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.747907 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.748800 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.749982 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.751909 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.752591 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.752751 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.757971 (+ 5220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:43875: BlockingRecv error: recv got EOF from 127.0.198.194:43875 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:04.538924 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.528147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:55227)
0807 19:45:04.529410 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.529435 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:04.529565 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:04.529887 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:04.529890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:04.530343 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:04.530498 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.530507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.531853 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.531857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.532840 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.532851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.533828 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.533833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.533973 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.534630 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:04.534649 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:04.538661 (+ 4012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":947,"threads_started":1}
W20260807 19:45:04.539332 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:04.528164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:55227 (local address 127.0.198.254:39291)
0807 19:45:04.528982 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:04.528994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:04.529513 (+ 519us) server_negotiation.cc:407] Connection header received
0807 19:45:04.529657 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:04.529661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:04.529732 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:04.529830 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:04.530718 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.531694 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.533037 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.533650 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.533789 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.538958 (+ 5169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:55227: BlockingRecv error: recv got EOF from 127.0.198.193:55227 (error 108)
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:04.679066 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.668514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:46983)
0807 19:45:04.669173 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.669222 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:04.669407 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:04.669817 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:04.669822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:04.670384 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:04.670645 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.670656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.671781 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.671788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.672770 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.672782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.673563 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.673570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.673769 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.674477 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:04.674496 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:04.678772 (+ 4276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":308,"threads_started":1}
W20260807 19:45:04.679560 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:04.668925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:46983 (local address 127.0.198.254:39291)
0807 19:45:04.669121 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:04.669140 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:04.669344 (+ 204us) server_negotiation.cc:407] Connection header received
0807 19:45:04.669579 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:04.669592 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:04.669661 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:04.669781 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:04.670806 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.671636 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.672926 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.673362 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.673509 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.679078 (+ 5569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:46983: BlockingRecv error: recv got EOF from 127.0.198.195:46983 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:04 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:04.770623 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.759722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:48175)
0807 19:45:04.760916 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.760973 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:04.761124 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:04.761576 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:04.761582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:04.762186 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:04.762341 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.762350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.763512 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.763516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.764553 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:04.764566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.765404 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.765408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:04.765566 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.766296 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:04.766319 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:04.770208 (+ 3889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":704,"threads_started":1}
W20260807 19:45:04.770716 10700 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:04.759890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:48175 (local address 127.0.198.254:39291)
0807 19:45:04.760206 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:04.760216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:04.761066 (+ 850us) server_negotiation.cc:407] Connection header received
0807 19:45:04.761290 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:04.761297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:04.761406 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:04.761521 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:04.762486 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.763291 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.764719 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.765187 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.765286 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.770439 (+ 5153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:48175: BlockingRecv error: recv got EOF from 127.0.198.194:48175 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:05.550777 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:05.539969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:34859)
0807 19:45:05.540858 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:05.540878 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:05.541042 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:05.541490 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:05.541494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:05.542032 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:05.542253 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.542265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.543574 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.543578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.544633 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.544648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.545541 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.545546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.545694 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.546441 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:05.546461 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:05.550436 (+ 3975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":549,"threads_started":1}
W20260807 19:45:05.551076 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:05.540063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:34859 (local address 127.0.198.254:39291)
0807 19:45:05.540752 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:05.540761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:05.540940 (+ 179us) server_negotiation.cc:407] Connection header received
0807 19:45:05.541209 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:05.541213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:05.541283 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:05.541386 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:05.542457 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.543319 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.544825 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.545353 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.545476 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.550762 (+ 5286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:34859: BlockingRecv error: recv got EOF from 127.0.198.193:34859 (error 108)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":405,"threads_started":1}
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:05.691622 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:05.680491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:60581)
0807 19:45:05.681132 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:05.681151 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:05.681329 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:05.681862 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:05.681871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:05.682513 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:05.682685 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.682695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.684084 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.684091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.685162 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.685176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.686022 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.686029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.686248 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.686930 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:05.686950 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:05.691197 (+ 4247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":339,"threads_started":1}
W20260807 19:45:05.691802 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:05.680788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:60581 (local address 127.0.198.254:39291)
0807 19:45:05.681034 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:05.681043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:05.681282 (+ 239us) server_negotiation.cc:407] Connection header received
0807 19:45:05.681540 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:05.681545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:05.681617 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:05.681739 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:05.682911 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.683915 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.685338 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.685835 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.685996 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.691537 (+ 5541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:60581: BlockingRecv error: recv got EOF from 127.0.198.195:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:05 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:05.782512 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:05.771965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:41375)
0807 19:45:05.772548 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:05.772581 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:05.772741 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:05.773355 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:05.773361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:05.773924 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:05.774117 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.774127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.775418 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.775425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.776581 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:05.776596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.777467 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.777472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:05.777629 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.778364 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:05.778383 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:05.782270 (+ 3887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":322,"threads_started":1}
W20260807 19:45:05.782924 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:05.772057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41375 (local address 127.0.198.254:39291)
0807 19:45:05.772262 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:05.772269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:05.772721 (+ 452us) server_negotiation.cc:407] Connection header received
0807 19:45:05.773013 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:05.773019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:05.773133 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:05.773265 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:05.774293 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.775168 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.776762 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:05.777274 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:05.777404 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:05.782537 (+ 5133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:41375: BlockingRecv error: recv got EOF from 127.0.198.194:41375 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:06.566392 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:06.552397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:48987)
0807 19:45:06.553000 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:06.553025 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:06.553196 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:06.553617 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:06.553621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:06.554089 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:06.554292 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.554301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.555557 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.555561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.556555 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.556568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.557346 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.557351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.557486 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.561755 (+ 4269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:06.561781 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:06.566043 (+ 4262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":320,"threads_started":1}
W20260807 19:45:06.566717 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:06.552545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:48987 (local address 127.0.198.254:39291)
0807 19:45:06.553182 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:06.553188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:06.553214 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:45:06.553323 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:06.553326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:06.553395 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:06.553566 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:06.554456 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.555396 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.556731 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.557177 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.557282 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.566328 (+ 9046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:48987: BlockingRecv error: recv got EOF from 127.0.198.193:48987 (error 108)
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":306,"threads_started":1}
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:06.703136 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:06.693011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:59419)
0807 19:45:06.693650 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:06.693667 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:06.693838 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:06.694227 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:06.694231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:06.694682 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:06.694845 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.694853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.696010 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.696016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.697006 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.697018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.697740 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.697746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.697925 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.698597 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:06.698615 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:06.702828 (+ 4213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":340,"threads_started":1}
W20260807 19:45:06.703696 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:06.693149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:59419 (local address 127.0.198.254:39291)
0807 19:45:06.693406 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:06.693416 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:06.693773 (+ 357us) server_negotiation.cc:407] Connection header received
0807 19:45:06.693991 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:06.693996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:06.694075 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:06.694190 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:06.695019 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.695851 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.697160 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.697580 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.697722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.703170 (+ 5448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:59419: BlockingRecv error: recv got EOF from 127.0.198.195:59419 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:06 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:06.793753 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:06.783798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:47909)
0807 19:45:06.784356 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:06.784410 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:06.784570 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:06.785022 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:06.785026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:06.785492 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:06.785708 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.785717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.786780 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.786785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.787778 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:06.787792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.788527 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.788531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:06.788660 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.789710 (+ 1050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:06.789735 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:06.793514 (+ 3779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":352,"threads_started":1}
W20260807 19:45:06.794364 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:06.783901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:47909 (local address 127.0.198.254:39291)
0807 19:45:06.784129 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:06.784139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:06.784512 (+ 373us) server_negotiation.cc:407] Connection header received
0807 19:45:06.784719 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:06.784725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:06.784840 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:06.784972 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:06.785845 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.786611 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.787936 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:06.788382 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:06.788483 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:06.794144 (+ 5661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:47909: BlockingRecv error: recv got EOF from 127.0.198.194:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:07.578615 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.567755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:33825)
0807 19:45:07.568392 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.568411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.568533 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.568926 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.568929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.569368 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.569558 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.569567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.570736 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.570740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.571899 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.571919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.573085 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.573090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.573300 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.574108 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.574128 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.578318 (+ 4190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":357,"threads_started":1}
W20260807 19:45:07.578891 10715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:07.567922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:33825 (local address 127.0.198.254:39291)
0807 19:45:07.568470 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.568475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.568494 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:07.568659 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.568663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.568730 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.568834 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.569729 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.570574 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.572066 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.572881 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.573062 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.578595 (+ 5533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:33825: BlockingRecv error: recv got EOF from 127.0.198.193:33825 (error 108)
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":310,"threads_started":1}
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:07.714612 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.704328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:37847)
0807 19:45:07.704844 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.704859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.705026 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.705433 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.705437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.705917 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.706089 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.706097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.707164 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.707171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.708167 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.708180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.708963 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.708969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.709162 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.709899 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.709921 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.714279 (+ 4358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":272,"threads_started":1}
W20260807 19:45:07.714912 10715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:07.704403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:37847 (local address 127.0.198.254:39291)
0807 19:45:07.704636 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.704644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.704953 (+ 309us) server_negotiation.cc:407] Connection header received
0807 19:45:07.705191 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.705195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.705277 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.705408 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.706247 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.707023 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.708335 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.708789 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.708944 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.714605 (+ 5661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:37847: BlockingRecv error: recv got EOF from 127.0.198.195:37847 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:07 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:07.805364 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.794871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:45051)
0807 19:45:07.795558 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.795579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.795747 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.796141 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.796147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.796611 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.796782 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.796791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.797856 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.797863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.798863 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.798875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.799739 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.799745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.799975 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.800647 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.800665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.804994 (+ 4329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":371,"threads_started":1}
W20260807 19:45:07.805600 10715 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:07.794941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:45051 (local address 127.0.198.254:39291)
0807 19:45:07.795172 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.795182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.795675 (+ 493us) server_negotiation.cc:407] Connection header received
0807 19:45:07.795890 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.795896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.795986 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.796099 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.796937 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.797693 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.799019 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.799541 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.799688 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.805310 (+ 5622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:45051: BlockingRecv error: recv got EOF from 127.0.198.194:45051 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:08.589747 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.579959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:55301)
0807 19:45:08.580521 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.580539 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.580644 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.581106 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.581110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.581515 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.581669 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.581677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.582791 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.582795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.583743 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.583754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.584495 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.584499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.584617 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.585248 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.585267 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.589389 (+ 4122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":320,"threads_started":1}
W20260807 19:45:08.589932 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:08.580012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:55301 (local address 127.0.198.254:39291)
0807 19:45:08.580618 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.580623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.580642 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:08.580795 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.580831 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.580926 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.581078 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.581821 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.582636 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.583899 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.584365 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.584464 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.589623 (+ 5159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:55301: BlockingRecv error: recv got EOF from 127.0.198.193:55301 (error 108)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":357,"threads_started":1}
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:08.725239 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.715858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:48127)
0807 19:45:08.716444 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.716461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.716577 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.716945 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.716950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.717441 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.717629 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.717639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.718692 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.718698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.719722 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.719734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.720499 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.720506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.720694 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.721339 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.721357 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.724872 (+ 3515us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":308,"threads_started":1}
W20260807 19:45:08.725417 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:08.715943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:48127 (local address 127.0.198.254:39291)
0807 19:45:08.716161 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.716168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.716544 (+ 376us) server_negotiation.cc:407] Connection header received
0807 19:45:08.716681 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.716685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.716770 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.716899 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.717780 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.718535 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.719860 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.720303 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.720436 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.725132 (+ 4696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:48127: BlockingRecv error: recv got EOF from 127.0.198.195:48127 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:08 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:08.816751 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.806635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:59015)
0807 19:45:08.807225 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.807247 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.807420 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.807852 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.807857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.808348 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.808600 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.808613 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.809792 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.809797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.810774 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.810787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.811526 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.811531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.811734 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.812363 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.812393 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.816390 (+ 3997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":342,"threads_started":1}
W20260807 19:45:08.816987 10720 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:08.806804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:59015 (local address 127.0.198.254:39291)
0807 19:45:08.806998 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.807002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.807331 (+ 329us) server_negotiation.cc:407] Connection header received
0807 19:45:08.807533 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.807537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.807665 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.807756 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.808786 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.809584 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.810930 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.811385 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.811487 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.816668 (+ 5181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:59015: BlockingRecv error: recv got EOF from 127.0.198.194:59015 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.193@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:09.602046 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:09.591360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.193:44251)
0807 19:45:09.591932 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:09.591950 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:09.592065 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:09.592642 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:09.592645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:09.593095 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:09.593285 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.593295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.594650 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.594654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.595805 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.595819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.596689 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.596694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.596844 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.597666 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:09.597686 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:09.601786 (+ 4100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
W20260807 19:45:09.602488 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:09.591512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:44251 (local address 127.0.198.254:39291)
0807 19:45:09.592350 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:09.592355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:09.592372 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:45:09.592426 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:09.592429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:09.592488 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:09.592588 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:09.593490 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.594469 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.595981 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.596504 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.596619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.602221 (+ 5602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.193:44251: BlockingRecv error: recv got EOF from 127.0.198.193:44251 (error 108)
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":563,"threads_started":1}
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.195@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:09.736951 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:09.726532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.195:52637)
0807 19:45:09.727147 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:09.727171 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:09.727295 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:09.727754 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:09.727758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:09.728224 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:09.728384 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.728392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.729600 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.729605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.730580 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.730594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.731601 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.731607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.731792 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.732511 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:09.732530 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:09.736538 (+ 4008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":343,"threads_started":1}
W20260807 19:45:09.737119 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:09.726611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:52637 (local address 127.0.198.254:39291)
0807 19:45:09.727025 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:09.727032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:09.727259 (+ 227us) server_negotiation.cc:407] Connection header received
0807 19:45:09.727447 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:09.727450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:09.727518 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:09.727623 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:09.728613 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.729432 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.730798 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.731408 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.731549 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.736794 (+ 5245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.195:52637: BlockingRecv error: recv got EOF from 127.0.198.195:52637 (error 108)
Metrics: {"server-negotiator.queue_time_us":249}
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:45:09 dist-test-slave-btw8 krb5kdc[9580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.198.194@KRBTEST.COM for oryx/127.0.198.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:45:09.829608 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:09.818150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39291 (local address 127.0.198.194:45339)
0807 19:45:09.818781 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:09.818804 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:09.818929 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:09.819622 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:09.819627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:09.820107 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:09.820354 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.820365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.821607 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.821612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.822796 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:09.822812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.823758 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.823763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:09.823911 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.824568 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:09.824586 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:09.829206 (+ 4620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.198.254:39291: Server oryx/127.0.198.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":295,"threads_started":1}
W20260807 19:45:09.829694 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:09.818345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:45339 (local address 127.0.198.254:39291)
0807 19:45:09.818534 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:09.818539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:09.818911 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:45:09.819067 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:09.819080 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:09.819253 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:09.819383 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:09.820538 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.821378 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.822993 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:09.823519 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:09.823700 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:09.829428 (+ 5728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.198.194:45339: BlockingRecv error: recv got EOF from 127.0.198.194:45339 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:45:10.317343 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10216
I20260807 19:45:10.333043 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:10.318754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.0.198.193:43621)
0807 19:45:10.319526 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:10.319534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:10.319628 (+ 94us) server_negotiation.cc:407] Connection header received
0807 19:45:10.319720 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:10.319728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:10.319991 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:10.320178 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:10.321734 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.322927 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.324412 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.325133 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.325267 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:10.329250 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:10.329283 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:10.329321 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:10.329402 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:10.332475 (+ 3073us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:45:10.332830 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45736: Request ticket server kudu/127.0.198.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":415,"threads_started":1}
W20260807 19:45:10.333194 10729 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45736: Request ticket server kudu/127.0.198.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:45:10.333369 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10216: Not authorized: Client connection negotiation failed: client connection to 127.0.198.193:43621: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:45:10.333715 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10216
I20260807 19:45:10.357249 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10379
I20260807 19:45:10.373633 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:10.358460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.0.198.194:35683)
0807 19:45:10.359185 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:10.359195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:10.359226 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:45:10.359306 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:10.359317 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:10.359664 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:10.359880 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:10.361858 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.363633 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.364986 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.365911 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.366061 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:10.369416 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:10.369426 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:10.369444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:10.369497 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:10.372956 (+ 3459us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:45:10.373342 (+ 386us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50772: Request ticket server kudu/127.0.198.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":325,"threads_started":1}
W20260807 19:45:10.373811 10730 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50772: Request ticket server kudu/127.0.198.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:45:10.373977 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10379: Not authorized: Client connection negotiation failed: client connection to 127.0.198.194:35683: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:45:10.374162 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10379
I20260807 19:45:10.395448 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10523
I20260807 19:45:10.410813 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:10.396751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.0.198.195:36575)
0807 19:45:10.397490 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:10.397500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:10.397533 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:45:10.397626 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:10.397635 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:10.397936 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:10.398148 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:10.399793 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.401061 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.402380 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:10.403216 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:10.403338 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:10.406375 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:10.406387 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:10.406410 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:10.406506 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:10.410167 (+ 3661us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:45:10.410541 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51296: Request ticket server kudu/127.0.198.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":361,"threads_started":1}
W20260807 19:45:10.410913 10731 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51296: Request ticket server kudu/127.0.198.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:45:10.411221 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10523: Not authorized: Client connection negotiation failed: client connection to 127.0.198.195:36575: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.198.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:45:10.411547 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10523
I20260807 19:45:10.432058 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10140
I20260807 19:45:10.433558 10209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:10.554438 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10140
2026-08-07T19:45:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22380 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T19:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:10Z Disabled control of system clock
I20260807 19:45:10.658591 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37903
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:45:11.028861 10736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:11.029209 10736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:11.029294 10736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:11.037117 10736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:11.037256 10736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:11.037297 10736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:11.037348 10736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:11.037395 10736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:11.047144 10736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:37903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:11.048633 10736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:11.050629 10736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:11.065155 10745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.065255 10744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.065129 10747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:11.065552 10736 server_base.cc:1034] running on GCE node
I20260807 19:45:11.066816 10736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:11.068568 10736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:11.069909 10736 hybrid_clock.cc:648] HybridClock initialized: now 1786131911069854 us; error 42 us; skew 500 ppm
I20260807 19:45:11.070387 10736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:11.073392 10736 webserver.cc:460] Webserver started at http://127.0.198.254:36087/ using document root <none> and password file <none>
I20260807 19:45:11.074218 10736 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:11.074328 10736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:11.074642 10736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:11.078105 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "3dfcbeeb60354d0780cd52a517bcc617"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "d30d04388d45c39c1625e9f2cfea526e"
server_key_iv: "46105a846b02123cf85198af4cf562e1"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.079005 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "3dfcbeeb60354d0780cd52a517bcc617"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "d30d04388d45c39c1625e9f2cfea526e"
server_key_iv: "46105a846b02123cf85198af4cf562e1"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.085000 10736 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 19:45:11.089457 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:11.091264 10736 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 19:45:11.091535 10736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "3dfcbeeb60354d0780cd52a517bcc617"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "d30d04388d45c39c1625e9f2cfea526e"
server_key_iv: "46105a846b02123cf85198af4cf562e1"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.091789 10736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:11.117527 10736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:11.118561 10736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:11.118848 10736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:11.140022 10736 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:37903
I20260807 19:45:11.140040 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:37903 every 8 connection(s)
I20260807 19:45:11.141785 10736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:11.142338 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10736
I20260807 19:45:11.142546 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:45:11.143437 795 external_mini_cluster.cc:1428] Setting key f9272e12a76fe9b63c0fc3d8e5c07844
I20260807 19:45:11.147878 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:11.160627 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:11.146000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.0.198.254:37903)
0807 19:45:11.147617 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:11.147638 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:11.147687 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:45:11.149564 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:11.149588 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:11.150093 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:11.150489 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:11.151331 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.152823 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.155060 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.155871 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.156023 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:11.157576 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:11.157584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:11.157599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:11.158190 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:11.158284 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:11.158311 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:11.158410 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:11.158693 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 19:45:11.159145 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":634,"threads_started":1}
I20260807 19:45:11.162825 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617: Bootstrap starting.
I20260807 19:45:11.166898 10811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:11.168550 10811 log.cc:824] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:11.172168 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617: No bootstrap required, opened a new log
I20260807 19:45:11.181728 10811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } }
I20260807 19:45:11.182152 10811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:11.182221 10811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dfcbeeb60354d0780cd52a517bcc617, State: Initialized, Role: FOLLOWER
I20260807 19:45:11.182767 10811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } }
I20260807 19:45:11.182976 10811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:11.183079 10811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:11.183215 10811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:11.185537 10811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } }
I20260807 19:45:11.186050 10811 leader_election.cc:302] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dfcbeeb60354d0780cd52a517bcc617; no voters:
I20260807 19:45:11.186710 10811 leader_election.cc:288] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:11.186883 10816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:11.187726 10816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [term 1 LEADER]: Becoming Leader. State: Replica: 3dfcbeeb60354d0780cd52a517bcc617, State: Running, Role: LEADER
I20260807 19:45:11.188531 10816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } }
I20260807 19:45:11.189368 10811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:11.192085 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dfcbeeb60354d0780cd52a517bcc617. Latest consensus state: current_term: 1 leader_uuid: "3dfcbeeb60354d0780cd52a517bcc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } } }
I20260807 19:45:11.192723 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:11.192178 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dfcbeeb60354d0780cd52a517bcc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfcbeeb60354d0780cd52a517bcc617" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37903 } } }
I20260807 19:45:11.193501 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:11.195896 10824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:11.203001 10824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:11.212406 10824 catalog_manager.cc:1347] Generated new cluster ID: edb2c3c1c4b948c38af0c7b69820d81c
I20260807 19:45:11.212567 10824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:11.233805 10824 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:11.235934 10824 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:11.247684 10824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dfcbeeb60354d0780cd52a517bcc617: Generated new TSK 0
I20260807 19:45:11.248709 10824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:45:11.262885 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--builtin_ntp_servers=127.0.198.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 19:45:11.652925 10835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:11.653270 10835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:11.653360 10835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:11.662106 10835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:11.662238 10835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:11.662353 10835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:45:11.673417 10835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:11.674872 10835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:11.676788 10835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:11.691433 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.691386 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.691519 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:11.692766 10835 server_base.cc:1034] running on GCE node
I20260807 19:45:11.693437 10835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:11.694475 10835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:11.695727 10835 hybrid_clock.cc:648] HybridClock initialized: now 1786131911695674 us; error 52 us; skew 500 ppm
I20260807 19:45:11.696162 10835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:11.702948 10835 webserver.cc:460] Webserver started at http://127.0.198.193:46881/ using document root <none> and password file <none>
I20260807 19:45:11.703867 10835 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:11.704025 10835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:11.704391 10835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:11.707584 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "b8f1a88f2ae14e6b8e91067e07482c6e"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "a1225ef151e673ff922fc8fb59970d70"
server_key_iv: "804e48a266cdf612f6ce553fb59a8476"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.708530 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "b8f1a88f2ae14e6b8e91067e07482c6e"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "a1225ef151e673ff922fc8fb59970d70"
server_key_iv: "804e48a266cdf612f6ce553fb59a8476"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.714156 10835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:45:11.718428 10849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:11.720008 10835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:11.720239 10835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "b8f1a88f2ae14e6b8e91067e07482c6e"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-btw8"
server_key: "a1225ef151e673ff922fc8fb59970d70"
server_key_iv: "804e48a266cdf612f6ce553fb59a8476"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.720508 10835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:11.743973 10835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:11.744976 10835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:11.745323 10835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:11.746433 10835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:11.748171 10835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:11.748307 10835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:11.748438 10835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:11.748534 10835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:11.786229 10835 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:37283
I20260807 19:45:11.786329 10973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:37283 every 8 connection(s)
I20260807 19:45:11.787740 10835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:11.794328 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10835
I20260807 19:45:11.794648 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:45:11.795571 795 external_mini_cluster.cc:1428] Setting key 8b0874db7bcc59d5b805e2d173bd275a
I20260807 19:45:11.799412 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--builtin_ntp_servers=127.0.198.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:45:11.804455 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:11.791009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:59837 (local address 127.0.198.254:37903)
0807 19:45:11.791724 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:11.791734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:11.793304 (+ 1570us) server_negotiation.cc:407] Connection header received
0807 19:45:11.796446 (+ 3142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:11.796455 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:11.796582 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:11.796745 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:11.798217 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.799094 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.800991 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.801436 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.801622 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:11.803291 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:11.803296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:11.803300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:11.803322 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:11.803420 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:11.803438 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:11.803660 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:11.803950 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 19:45:11.804070 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":265,"threads_started":1}
I20260807 19:45:11.805614 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:11.791832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37903 (local address 127.0.198.193:59837)
0807 19:45:11.793086 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:11.793136 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:11.795953 (+ 2817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:11.797090 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:11.797100 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:11.797216 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:11.797999 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:11.798029 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.799288 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.799292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:11.800796 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:11.800813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:11.801739 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:11.801748 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:11.802020 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:11.802882 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:11.802885 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:11.802933 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:11.803043 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:11.803638 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:11.803660 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:11.803671 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:45:11.803939 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:45:11.804423 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":351,"threads_started":1}
I20260807 19:45:11.807448 10974 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37903
I20260807 19:45:11.807834 10974 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:11.808864 10974 heartbeater.cc:511] Master 127.0.198.254:37903 requested a full tablet report, sending...
I20260807 19:45:11.811625 10774 ts_manager.cc:194] Registered new tserver with Master: b8f1a88f2ae14e6b8e91067e07482c6e (127.0.198.193:37283)
I20260807 19:45:11.813309 10774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.193:59837
W20260807 19:45:12.185474 10978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:12.185796 10978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:12.185940 10978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:12.194048 10978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:12.194176 10978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:12.194362 10978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:45:12.204705 10978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:12.206151 10978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:12.208063 10978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:12.223464 10984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:12.223537 10986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:12.223500 10983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:12.224134 10978 server_base.cc:1034] running on GCE node
I20260807 19:45:12.225060 10978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:12.226254 10978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:12.227496 10978 hybrid_clock.cc:648] HybridClock initialized: now 1786131912227456 us; error 38 us; skew 500 ppm
I20260807 19:45:12.227897 10978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:12.230829 10978 webserver.cc:460] Webserver started at http://127.0.198.194:34305/ using document root <none> and password file <none>
I20260807 19:45:12.231772 10978 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:12.231927 10978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:12.232292 10978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:12.235512 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "53942d45606d4cc09b871b987cec3a27"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "16adbc0edaee4ce862d0f994592e2558"
server_key_iv: "f1cfbeb22c418f7d770fc2da1b34703c"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.236476 10978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "53942d45606d4cc09b871b987cec3a27"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "16adbc0edaee4ce862d0f994592e2558"
server_key_iv: "f1cfbeb22c418f7d770fc2da1b34703c"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.242478 10978 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:45:12.246670 10992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.248474 10978 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:12.248704 10978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "53942d45606d4cc09b871b987cec3a27"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "16adbc0edaee4ce862d0f994592e2558"
server_key_iv: "f1cfbeb22c418f7d770fc2da1b34703c"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.248983 10978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:12.274482 10978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:12.275560 10978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:12.275904 10978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:12.277034 10978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:12.278731 10978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:12.278875 10978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.279026 10978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:12.279125 10978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.317436 10978 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:38437
I20260807 19:45:12.317544 11112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:38437 every 8 connection(s)
I20260807 19:45:12.318926 10978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:12.327064 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 10978
I20260807 19:45:12.327281 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:45:12.327780 795 external_mini_cluster.cc:1428] Setting key 3c879624f0c466c248fad3be73040f72
I20260807 19:45:12.331028 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--builtin_ntp_servers=127.0.198.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:45:12.334161 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.322009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:59435 (local address 127.0.198.254:37903)
0807 19:45:12.322532 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.322540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.324023 (+ 1483us) server_negotiation.cc:407] Connection header received
0807 19:45:12.326080 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.326086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:12.326201 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:12.326331 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:12.327677 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.328642 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.330867 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.331461 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.331606 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.333222 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:12.333227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:12.333231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:12.333253 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:12.333296 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:12.333310 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:12.333414 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:12.333676 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 19:45:12.333794 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":248,"threads_started":1}
I20260807 19:45:12.335441 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.322725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37903 (local address 127.0.198.194:59435)
0807 19:45:12.323829 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:12.323875 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:12.325707 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:12.326617 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:12.326626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:12.326742 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:12.327488 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:12.327510 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.328824 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.328834 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:12.330705 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:12.330719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.331737 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.331749 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:12.332005 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.332822 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:12.332825 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:12.332873 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:12.332972 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:12.333415 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:12.333429 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:12.333442 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 19:45:12.333662 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 19:45:12.334156 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":339,"threads_started":1}
I20260807 19:45:12.337425 11113 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37903
I20260807 19:45:12.337909 11113 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:12.338860 11113 heartbeater.cc:511] Master 127.0.198.254:37903 requested a full tablet report, sending...
I20260807 19:45:12.340833 10774 ts_manager.cc:194] Registered new tserver with Master: 53942d45606d4cc09b871b987cec3a27 (127.0.198.194:38437)
I20260807 19:45:12.342109 10774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.194:59435
W20260807 19:45:12.696925 11117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:12.697268 11117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:12.697361 11117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:12.705080 11117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:12.705199 11117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:12.705312 11117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:45:12.715274 11117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:12.716809 11117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:12.718791 11117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:12.733855 11122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:12.733971 11125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:12.733925 11123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:12.734992 11117 server_base.cc:1034] running on GCE node
I20260807 19:45:12.735983 11117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:12.737181 11117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:12.738451 11117 hybrid_clock.cc:648] HybridClock initialized: now 1786131912738400 us; error 41 us; skew 500 ppm
I20260807 19:45:12.738832 11117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:12.741822 11117 webserver.cc:460] Webserver started at http://127.0.198.195:39351/ using document root <none> and password file <none>
I20260807 19:45:12.742614 11117 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:12.742733 11117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:12.743037 11117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:12.746630 11117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "582ada704de74cb69ca42654c2f26511"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "c020aca48bd3becd5023b6d7b05821f2"
server_key_iv: "8282ef9dbdd911739507667dffa94ae4"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.747675 11117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "582ada704de74cb69ca42654c2f26511"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "c020aca48bd3becd5023b6d7b05821f2"
server_key_iv: "8282ef9dbdd911739507667dffa94ae4"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.754350 11117 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:45:12.758570 11132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.760226 11117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:12.760449 11117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "582ada704de74cb69ca42654c2f26511"
format_stamp: "Formatted at 2026-08-07 19:45:12 on dist-test-slave-btw8"
server_key: "c020aca48bd3becd5023b6d7b05821f2"
server_key_iv: "8282ef9dbdd911739507667dffa94ae4"
server_key_version: "encryptionkey@0"
I20260807 19:45:12.760751 11117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:12.796684 11117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:12.797794 11117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:12.798151 11117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:12.799229 11117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:12.800928 11117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:12.801061 11117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.801221 11117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:12.801318 11117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:12.817041 10974 heartbeater.cc:503] Master 127.0.198.254:37903 was elected leader, sending a full tablet report...
I20260807 19:45:12.839273 11117 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:43367
I20260807 19:45:12.839407 11251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:43367 every 8 connection(s)
I20260807 19:45:12.840703 11117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:12.846756 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11117
I20260807 19:45:12.846966 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:12.847491 795 external_mini_cluster.cc:1428] Setting key ea0a868ea1f994e77a099cfd9a720bd8
I20260807 19:45:12.855706 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.843677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:52537 (local address 127.0.198.254:37903)
0807 19:45:12.844162 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.844167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.845519 (+ 1352us) server_negotiation.cc:407] Connection header received
0807 19:45:12.847868 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.847874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:12.847982 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:12.848154 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:12.849772 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.850697 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.852550 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.853162 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.853291 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.854778 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:12.854783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:12.854786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:12.854808 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:12.854852 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:12.854865 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:12.854946 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:12.855206 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 19:45:12.855297 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":231,"threads_started":1}
I20260807 19:45:12.856720 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.844315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37903 (local address 127.0.198.195:52537)
0807 19:45:12.845319 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:12.845359 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:12.847488 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:12.848457 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:12.848470 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:12.848626 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:12.849542 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:12.849578 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.850878 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.850882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:12.852388 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:12.852402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.853400 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.853405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:12.853625 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.854428 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:12.854431 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:12.854475 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:12.854562 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:12.854961 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:12.854969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:12.854975 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:45:12.855128 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 19:45:12.855531 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":341,"threads_started":1}
I20260807 19:45:12.858247 11252 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37903
I20260807 19:45:12.858579 11252 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:12.859467 11252 heartbeater.cc:511] Master 127.0.198.254:37903 requested a full tablet report, sending...
I20260807 19:45:12.860989 10774 ts_manager.cc:194] Registered new tserver with Master: 582ada704de74cb69ca42654c2f26511 (127.0.198.195:43367)
I20260807 19:45:12.862121 10774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.195:52537
I20260807 19:45:12.863891 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:45:12.870721 11254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:12.869038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.0.198.254:37903)
0807 19:45:12.869327 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.869335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.869667 (+ 332us) server_negotiation.cc:407] Connection header received
0807 19:45:12.869901 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.869906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:12.869972 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:12.870081 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:12.870526 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42472: BlockingRecv error: recv got EOF from 127.0.0.1:42472 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 19:45:12.871984 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10835
I20260807 19:45:12.880085 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.873012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.0.198.193:37283)
0807 19:45:12.873632 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.873639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.873666 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:45:12.873724 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.873730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:12.873939 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:12.874079 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:12.874461 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.875564 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.876847 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.877426 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.877521 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.878645 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:12.878655 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:12.878668 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:12.879272 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:12.879325 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:12.879404 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:12.879518 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:12.879646 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:45:12.879772 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":322,"threads_started":1}
I20260807 19:45:12.881457 10969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:12.977048 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10835
I20260807 19:45:12.997895 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10978
I20260807 19:45:13.007464 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.999084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.0.198.194:38437)
0807 19:45:12.999726 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.999734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.999763 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:45:12.999828 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.999835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:13.000060 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:13.000202 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:13.000688 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:13.001825 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:13.003322 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:13.004142 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:13.004267 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:13.005557 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:13.005569 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:13.005583 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:13.006469 (+ 886us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:13.006527 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:13.006552 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:13.006654 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:13.006878 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:45:13.007028 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":359,"threads_started":1}
I20260807 19:45:13.009034 11108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:13.094403 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10978
I20260807 19:45:13.114466 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11117
I20260807 19:45:13.123659 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:13.115698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.0.198.195:43367)
0807 19:45:13.116493 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:13.116510 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:13.116554 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:45:13.116623 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:13.116630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:13.116904 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:13.117065 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:13.117572 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:13.118675 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:13.120164 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:13.120798 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:13.120908 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:13.122030 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:13.122038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:13.122053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:13.122666 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:13.122726 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:13.122747 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:13.122851 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:13.123021 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 19:45:13.123152 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":393,"threads_started":1}
I20260807 19:45:13.125135 11247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:13.212657 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11117
I20260807 19:45:13.232769 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 10736
I20260807 19:45:13.234069 10805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:13.345170 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 10736
2026-08-07T19:45:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2784 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T19:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:13Z Disabled control of system clock
I20260807 19:45:13.445608 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36727
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:36727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:45:13.817082 11273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:13.817421 11273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:13.817510 11273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:13.825361 11273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:13.825515 11273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:13.825555 11273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:13.825609 11273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:13.825654 11273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:13.835538 11273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:36727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:13.836984 11273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:13.838840 11273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:13.852440 11278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:13.852393 11281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:13.852353 11279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:13.853433 11273 server_base.cc:1034] running on GCE node
I20260807 19:45:13.854414 11273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:13.856452 11273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:13.857750 11273 hybrid_clock.cc:648] HybridClock initialized: now 1786131913857709 us; error 43 us; skew 500 ppm
I20260807 19:45:13.858211 11273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:13.861099 11273 webserver.cc:460] Webserver started at http://127.0.198.254:35743/ using document root <none> and password file <none>
I20260807 19:45:13.861953 11273 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:13.862115 11273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:13.862524 11273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:13.866007 11273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "e1edc056e060466dabb9552aeca750e2"
format_stamp: "Formatted at 2026-08-07 19:45:13 on dist-test-slave-btw8"
server_key: "c58309991108b46f90afbc9b54201cea"
server_key_iv: "cd1a84ef4959d40a3e2305d42152e4d9"
server_key_version: "encryptionkey@0"
I20260807 19:45:13.867017 11273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "e1edc056e060466dabb9552aeca750e2"
format_stamp: "Formatted at 2026-08-07 19:45:13 on dist-test-slave-btw8"
server_key: "c58309991108b46f90afbc9b54201cea"
server_key_iv: "cd1a84ef4959d40a3e2305d42152e4d9"
server_key_version: "encryptionkey@0"
I20260807 19:45:13.873371 11273 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:45:13.877676 11288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:13.879480 11273 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:13.879782 11273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "e1edc056e060466dabb9552aeca750e2"
format_stamp: "Formatted at 2026-08-07 19:45:13 on dist-test-slave-btw8"
server_key: "c58309991108b46f90afbc9b54201cea"
server_key_iv: "cd1a84ef4959d40a3e2305d42152e4d9"
server_key_version: "encryptionkey@0"
I20260807 19:45:13.880090 11273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:13.886480 11273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:13.887516 11273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:13.887876 11273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:13.910840 11273 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:36727
I20260807 19:45:13.911008 11343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:36727 every 8 connection(s)
I20260807 19:45:13.912464 11273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:13.917004 11344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:13.921584 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11273
I20260807 19:45:13.921854 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:45:13.922617 795 external_mini_cluster.cc:1428] Setting key efa923b33b229e45ba8596b17e0a36c0
I20260807 19:45:13.934985 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2: Bootstrap starting.
I20260807 19:45:13.938380 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:13.925007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.0.198.254:36727)
0807 19:45:13.926192 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:13.926205 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:13.932205 (+ 6000us) server_negotiation.cc:407] Connection header received
0807 19:45:13.934279 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:13.934305 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:13.934841 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:13.935287 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:13.936552 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:13.936559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:13.936572 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:13.936589 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:13.936651 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:13.936669 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:13.936969 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:13.937118 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 19:45:13.937395 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":436,"threads_started":1}
I20260807 19:45:13.939195 11344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:13.940529 11344 log.cc:824] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:13.943847 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2: No bootstrap required, opened a new log
I20260807 19:45:13.952225 11344 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } }
I20260807 19:45:13.952718 11344 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:13.952857 11344 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1edc056e060466dabb9552aeca750e2, State: Initialized, Role: FOLLOWER
I20260807 19:45:13.953610 11344 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } }
I20260807 19:45:13.953922 11344 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:13.954141 11344 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:13.954391 11344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:13.957150 11344 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } }
I20260807 19:45:13.957789 11344 leader_election.cc:302] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1edc056e060466dabb9552aeca750e2; no voters:
I20260807 19:45:13.958606 11344 leader_election.cc:288] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:13.958810 11349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:13.960083 11349 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [term 1 LEADER]: Becoming Leader. State: Replica: e1edc056e060466dabb9552aeca750e2, State: Running, Role: LEADER
I20260807 19:45:13.960680 11349 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } }
I20260807 19:45:13.961272 11344 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:13.963698 11351 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1edc056e060466dabb9552aeca750e2. Latest consensus state: current_term: 1 leader_uuid: "e1edc056e060466dabb9552aeca750e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } } }
I20260807 19:45:13.964478 11351 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:13.964272 11350 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1edc056e060466dabb9552aeca750e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1edc056e060466dabb9552aeca750e2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36727 } } }
I20260807 19:45:13.965057 11350 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:13.966941 11359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:13.973140 11359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:13.984656 11359 catalog_manager.cc:1347] Generated new cluster ID: 0bf61b4e302b4de2837c3e553031fe43
I20260807 19:45:13.984910 11359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:14.015823 11359 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:14.016357 11359 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:14.032534 11359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1edc056e060466dabb9552aeca750e2: Generated new TSK 0
I20260807 19:45:14.033674 11359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:45:14.051847 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--builtin_ntp_servers=127.0.198.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:45:14.425129 11369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:14.425519 11369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:14.425683 11369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:14.434445 11369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:14.434644 11369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:14.434827 11369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:45:14.445812 11369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:14.447302 11369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:14.449329 11369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:14.463786 11374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.463768 11377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.463696 11375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:14.465051 11369 server_base.cc:1034] running on GCE node
I20260807 19:45:14.465823 11369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:14.467090 11369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:14.468350 11369 hybrid_clock.cc:648] HybridClock initialized: now 1786131914468268 us; error 66 us; skew 500 ppm
I20260807 19:45:14.468777 11369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:14.471756 11369 webserver.cc:460] Webserver started at http://127.0.198.193:33551/ using document root <none> and password file <none>
I20260807 19:45:14.472574 11369 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:14.472682 11369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:14.473100 11369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:14.476657 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "7256f6ffe1114b93b4ffb1efdd53dccd"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "e66c35464538ed6006d2280a40c30eb2"
server_key_iv: "6706e3de03b420483da8fd13c9464717"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.477617 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "7256f6ffe1114b93b4ffb1efdd53dccd"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "e66c35464538ed6006d2280a40c30eb2"
server_key_iv: "6706e3de03b420483da8fd13c9464717"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.483897 11369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 19:45:14.488261 11384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.490043 11369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:45:14.490276 11369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "7256f6ffe1114b93b4ffb1efdd53dccd"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "e66c35464538ed6006d2280a40c30eb2"
server_key_iv: "6706e3de03b420483da8fd13c9464717"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.490547 11369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:14.497737 11369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:14.498714 11369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:14.499038 11369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:14.500272 11369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:14.502050 11369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:14.502183 11369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.502334 11369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:14.502439 11369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.542320 11369 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:44813
I20260807 19:45:14.542450 11504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:44813 every 8 connection(s)
I20260807 19:45:14.543815 11369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:14.546497 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11369
I20260807 19:45:14.546777 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:45:14.547633 795 external_mini_cluster.cc:1428] Setting key cc461f6c6f12c74a2cf802206ae92498
I20260807 19:45:14.551156 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--builtin_ntp_servers=127.0.198.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:45:14.554291 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:14.546942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:56993 (local address 127.0.198.254:36727)
0807 19:45:14.547554 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:14.547562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:14.549029 (+ 1467us) server_negotiation.cc:407] Connection header received
0807 19:45:14.551446 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:14.551457 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:14.551536 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:14.551737 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:14.553483 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:14.553488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:14.553492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:14.553511 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:14.553565 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:14.553579 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:14.553711 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:14.553947 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 19:45:14.554005 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":272,"threads_started":1}
I20260807 19:45:14.555644 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:14.547625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36727 (local address 127.0.198.193:56993)
0807 19:45:14.548842 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:14.548893 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:14.551030 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:14.552024 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:14.552039 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:14.552171 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:14.553063 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:14.553070 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:14.553128 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:14.553251 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:14.553714 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:14.553727 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:14.553738 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:45:14.554023 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 19:45:14.554308 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":344,"threads_started":1}
I20260807 19:45:14.557312 11505 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36727
I20260807 19:45:14.557693 11505 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:14.558328 11505 heartbeater.cc:511] Master 127.0.198.254:36727 requested a full tablet report, sending...
I20260807 19:45:14.560499 11307 ts_manager.cc:194] Registered new tserver with Master: 7256f6ffe1114b93b4ffb1efdd53dccd (127.0.198.193:44813)
W20260807 19:45:14.923249 11510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:14.923836 11510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:14.924038 11510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:14.932775 11510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:14.932914 11510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:14.933096 11510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:45:14.943544 11510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:14.945104 11510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:14.947053 11510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:14.962374 11515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.962477 11516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.962466 11518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:14.963891 11510 server_base.cc:1034] running on GCE node
I20260807 19:45:14.964732 11510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:14.966045 11510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:14.967307 11510 hybrid_clock.cc:648] HybridClock initialized: now 1786131914967258 us; error 52 us; skew 500 ppm
I20260807 19:45:14.967876 11510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:14.970736 11510 webserver.cc:460] Webserver started at http://127.0.198.194:43217/ using document root <none> and password file <none>
I20260807 19:45:14.971691 11510 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:14.971803 11510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:14.972196 11510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:14.975586 11510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "003e0743335b4638a05a493e3b309731"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "0888d7cf58065da13ee8e77eeec4d6b7"
server_key_iv: "7b358ab460cc3bdff45e1bd037952444"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.976531 11510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "003e0743335b4638a05a493e3b309731"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "0888d7cf58065da13ee8e77eeec4d6b7"
server_key_iv: "7b358ab460cc3bdff45e1bd037952444"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.982704 11510 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:45:14.987187 11524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.988849 11510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:45:14.989064 11510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "003e0743335b4638a05a493e3b309731"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-btw8"
server_key: "0888d7cf58065da13ee8e77eeec4d6b7"
server_key_iv: "7b358ab460cc3bdff45e1bd037952444"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.989327 11510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:14.996924 11510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:14.997941 11510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:14.998333 11510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:14.999501 11510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:15.001128 11510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:15.001267 11510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.001430 11510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:15.001523 11510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.041656 11510 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35905
I20260807 19:45:15.041725 11641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35905 every 8 connection(s)
I20260807 19:45:15.043278 11510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:15.047434 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11510
I20260807 19:45:15.047683 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:45:15.048285 795 external_mini_cluster.cc:1428] Setting key 22a2fde5722c778b14c2cd54c4eefc9d
I20260807 19:45:15.051856 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--builtin_ntp_servers=127.0.198.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:45:15.053642 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.046582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:45001 (local address 127.0.198.254:36727)
0807 19:45:15.046824 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.046831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.048707 (+ 1876us) server_negotiation.cc:407] Connection header received
0807 19:45:15.050778 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.050784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.050852 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.050973 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.052596 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.052604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.052611 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:15.052637 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.052683 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:15.052712 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.052857 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.053187 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.053266 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 19:45:15.054767 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.047182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36727 (local address 127.0.198.194:45001)
0807 19:45:15.048477 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.048538 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.050475 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.051194 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.051203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.051321 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:15.052273 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:15.052276 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:15.052316 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:15.052409 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:15.052902 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:15.052914 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:15.052929 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.053251 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.053497 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":836,"threads_started":1}
I20260807 19:45:15.056706 11642 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36727
I20260807 19:45:15.057102 11642 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:15.057828 11642 heartbeater.cc:511] Master 127.0.198.254:36727 requested a full tablet report, sending...
I20260807 19:45:15.059675 11307 ts_manager.cc:194] Registered new tserver with Master: 003e0743335b4638a05a493e3b309731 (127.0.198.194:35905)
W20260807 19:45:15.427500 11645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:15.427858 11645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:15.427960 11645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:15.435698 11645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:15.435937 11645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:15.436120 11645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:45:15.446008 11645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:15.447566 11645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:15.449554 11645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:15.464614 11650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.464550 11651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.464571 11654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:15.466020 11645 server_base.cc:1034] running on GCE node
I20260807 19:45:15.466861 11645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:15.468175 11645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:15.469430 11645 hybrid_clock.cc:648] HybridClock initialized: now 1786131915469362 us; error 55 us; skew 500 ppm
I20260807 19:45:15.469854 11645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:15.472774 11645 webserver.cc:460] Webserver started at http://127.0.198.195:32975/ using document root <none> and password file <none>
I20260807 19:45:15.473655 11645 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:15.473759 11645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:15.474109 11645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:15.477407 11645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "18fe4546ce6d4bc6a157347723ff6367"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-btw8"
server_key: "b5261a6a9c1dfa35313972b6a3a1648e"
server_key_iv: "92d119125f06f73ea5f44a6bab5a44fe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.478335 11645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "18fe4546ce6d4bc6a157347723ff6367"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-btw8"
server_key: "b5261a6a9c1dfa35313972b6a3a1648e"
server_key_iv: "92d119125f06f73ea5f44a6bab5a44fe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.484576 11645 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:45:15.489171 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.490844 11645 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:45:15.491077 11645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "18fe4546ce6d4bc6a157347723ff6367"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-btw8"
server_key: "b5261a6a9c1dfa35313972b6a3a1648e"
server_key_iv: "92d119125f06f73ea5f44a6bab5a44fe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.491466 11645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:15.498281 11645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:15.499173 11645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:15.499571 11645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:15.500597 11645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:15.502172 11645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:15.502270 11645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.502424 11645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:15.502528 11645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.541649 11645 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:33797
I20260807 19:45:15.541749 11781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:33797 every 8 connection(s)
I20260807 19:45:15.543212 11645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:15.548821 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11645
I20260807 19:45:15.549052 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:15.549628 795 external_mini_cluster.cc:1428] Setting key 9f0c3040b637d01f1b13589c898b4ea4
I20260807 19:45:15.553435 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.546452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:45277 (local address 127.0.198.254:36727)
0807 19:45:15.546697 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.546704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.548249 (+ 1545us) server_negotiation.cc:407] Connection header received
0807 19:45:15.550775 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.550782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.550854 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.550973 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.552714 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.552722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.552726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:15.552748 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.552788 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:15.552812 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.552932 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.553167 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.553224 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:45:15.554490 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.546989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:36727 (local address 127.0.198.195:45277)
0807 19:45:15.548073 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.548116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.550369 (+ 2253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.551333 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.551346 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.551532 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:15.552407 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:45:15.552410 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:15.552449 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:45:15.552540 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:15.552924 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:15.552937 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:15.552945 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.553257 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.553508 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":369,"threads_started":1}
I20260807 19:45:15.555907 11782 heartbeater.cc:347] Connected to a master server at 127.0.198.254:36727
I20260807 19:45:15.556231 11782 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:15.556910 11782 heartbeater.cc:511] Master 127.0.198.254:36727 requested a full tablet report, sending...
I20260807 19:45:15.558465 11307 ts_manager.cc:194] Registered new tserver with Master: 18fe4546ce6d4bc6a157347723ff6367 (127.0.198.195:33797)
I20260807 19:45:15.563640 11505 heartbeater.cc:503] Master 127.0.198.254:36727 was elected leader, sending a full tablet report...
I20260807 19:45:15.566452 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:45:15.573427 11507 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:45:15.571855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54398 (local address 127.0.198.254:36727)
0807 19:45:15.572075 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.572081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.572569 (+ 488us) server_negotiation.cc:407] Connection header received
0807 19:45:15.572703 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.572708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.572766 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.572862 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.573210 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54398: BlockingRecv error: recv got EOF from 127.0.0.1:54398 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:45:15.574164 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11369
I20260807 19:45:15.578076 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.575258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.0.198.193:44813)
0807 19:45:15.575923 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.575930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.575958 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:45:15.576020 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.576027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.576240 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.576389 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.577415 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.577421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.577434 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:15.577447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.577493 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:15.577507 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.577647 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.577694 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.577785 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":351,"threads_started":1}
I20260807 19:45:15.579447 11500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:15.665436 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11369
I20260807 19:45:15.685110 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11510
I20260807 19:45:15.689335 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.686298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.0.198.194:35905)
0807 19:45:15.686809 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.686816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.686842 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:45:15.686905 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.686911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.687132 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.687288 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.688380 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.688388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.688400 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:15.688426 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.688477 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:15.688493 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.688651 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.688891 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.688990 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":235,"threads_started":1}
I20260807 19:45:15.690752 11637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:15.778276 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11510
I20260807 19:45:15.797668 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11645
I20260807 19:45:15.802059 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.798839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.0.198.195:33797)
0807 19:45:15.799638 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.799646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.799679 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:45:15.799745 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.799751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.800038 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.800195 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.801295 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.801301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.801312 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:45:15.801339 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.801382 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:45:15.801396 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.801534 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.801589 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.801708 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":401,"threads_started":1}
I20260807 19:45:15.803476 11777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:15.889493 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11645
I20260807 19:45:15.922713 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11273
I20260807 19:45:15.924186 11339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:16.035338 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11273
2026-08-07T19:45:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2690 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:16 dist-test-slave-btw8 krb5kdc[11803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:16 dist-test-slave-btw8 krb5kdc[11803](info): set up 2 sockets
Aug 07 19:45:16 dist-test-slave-btw8 krb5kdc[11803](info): commencing operation
krb5kdc: starting...
W20260807 19:45:17.249653 795 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:45:17 dist-test-slave-btw8 krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:45:17.596433 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34253
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:17.967967 11816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:17.968302 11816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:17.968468 11816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:17.976392 11816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:17.976532 11816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:17.976627 11816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:17.976711 11816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:17.976775 11816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:17.987478 11816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:17.988991 11816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:17.990434 11816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:18.004041 11822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.004153 11824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.004594 11821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:18.005053 11816 server_base.cc:1034] running on GCE node
I20260807 19:45:18.005981 11816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:18.007876 11816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:18.009186 11816 hybrid_clock.cc:648] HybridClock initialized: now 1786131918009097 us; error 68 us; skew 500 ppm
I20260807 19:45:18.009641 11816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:18 dist-test-slave-btw8 krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:18.019202 11816 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:45:18.021710 11816 webserver.cc:460] Webserver started at http://127.0.198.254:43103/ using document root <none> and password file <none>
I20260807 19:45:18.022483 11816 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:18.022679 11816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:18.022977 11816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:18.026726 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "63406ccb3c014d93b25ca853a6ebd37e"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "88341f70fe48cf14934c5ae08a7a60ee"
server_key_iv: "4a5c36a151c2310b8815d85e191636a5"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.027746 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "63406ccb3c014d93b25ca853a6ebd37e"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "88341f70fe48cf14934c5ae08a7a60ee"
server_key_iv: "4a5c36a151c2310b8815d85e191636a5"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.034276 11816 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:45:18.039081 11831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.041074 11816 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:18.041344 11816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "63406ccb3c014d93b25ca853a6ebd37e"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "88341f70fe48cf14934c5ae08a7a60ee"
server_key_iv: "4a5c36a151c2310b8815d85e191636a5"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.041618 11816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:18.062775 11816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:18.068050 11816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:18.068392 11816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:18.090318 11816 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:34253
I20260807 19:45:18.090349 11884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:34253 every 8 connection(s)
I20260807 19:45:18.092453 11816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:18.097100 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:18.102169 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11816
I20260807 19:45:18.102455 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:45:18.103152 795 external_mini_cluster.cc:1428] Setting key a21e355ad462e53eb96670caa0504ac4
I20260807 19:45:18.109659 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: Bootstrap starting.
I20260807 19:45:18.113938 11885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:18.115509 11885 log.cc:824] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:18.119988 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: No bootstrap required, opened a new log
Aug 07 19:45:18 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:45:18.129132 11885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } }
I20260807 19:45:18.129606 11885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:18.129758 11885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63406ccb3c014d93b25ca853a6ebd37e, State: Initialized, Role: FOLLOWER
I20260807 19:45:18.130548 11885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } }
I20260807 19:45:18.130862 11885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:18.131079 11885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:18.131379 11885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:18.133441 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.105722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.0.198.254:34253)
0807 19:45:18.106688 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.106702 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.106750 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:45:18.108789 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.108818 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.109265 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.109740 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.111871 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.113260 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.115433 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.116663 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.116921 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.122566 (+ 5645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.122578 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.122602 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.122664 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.127300 (+ 4636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.128468 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.128480 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.128489 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.128744 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.129294 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.129302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.129315 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.130275 (+ 960us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.130666 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.131225 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.131837 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":447,"threads_started":1}
I20260807 19:45:18.134729 11885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } }
I20260807 19:45:18.135329 11885 leader_election.cc:302] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63406ccb3c014d93b25ca853a6ebd37e; no voters:
I20260807 19:45:18.136093 11885 leader_election.cc:288] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:18.136287 11890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:18.137944 11890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [term 1 LEADER]: Becoming Leader. State: Replica: 63406ccb3c014d93b25ca853a6ebd37e, State: Running, Role: LEADER
I20260807 19:45:18.138556 11885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:18.138525 11890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } }
I20260807 19:45:18.142062 11891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63406ccb3c014d93b25ca853a6ebd37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } } }
I20260807 19:45:18.142464 11891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:18.142910 11892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63406ccb3c014d93b25ca853a6ebd37e. Latest consensus state: current_term: 1 leader_uuid: "63406ccb3c014d93b25ca853a6ebd37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63406ccb3c014d93b25ca853a6ebd37e" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34253 } } }
I20260807 19:45:18.143270 11892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:18.146302 11898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:18.152661 11898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:18.162688 11898 catalog_manager.cc:1347] Generated new cluster ID: 63c559388aa84144b8a610881eba0a7a
I20260807 19:45:18.162837 11898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:18.174276 11898 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:18.176307 11898 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:18.188182 11898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: Generated new TSK 0
I20260807 19:45:18.189394 11898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:45:18.287570 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:18.671931 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:18.672288 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:18.672457 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:18.680276 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:18.680437 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:18.680625 11911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:45:18.692088 11911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:18.693526 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:18.694892 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:18.709241 11918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.709667 11917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.710033 11920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:18.710428 11911 server_base.cc:1034] running on GCE node
I20260807 19:45:18.711161 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:18.712306 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:18.713569 11911 hybrid_clock.cc:648] HybridClock initialized: now 1786131918713510 us; error 54 us; skew 500 ppm
I20260807 19:45:18.713940 11911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:18 dist-test-slave-btw8 krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:18.717700 11911 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:45:18.719547 11911 webserver.cc:460] Webserver started at http://127.0.198.193:40053/ using document root <none> and password file <none>
I20260807 19:45:18.720335 11911 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:18.720496 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:18.720888 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:18.724512 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "c8594a5f05a8433580f881986cf05b22"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "3df0a654fc0b0068b2e69e07d47398aa"
server_key_iv: "6edc2da3bae8912f477b95ed892c09ad"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.725482 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "c8594a5f05a8433580f881986cf05b22"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "3df0a654fc0b0068b2e69e07d47398aa"
server_key_iv: "6edc2da3bae8912f477b95ed892c09ad"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.731922 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:45:18.736508 11927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.738703 11911 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 19:45:18.739013 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "c8594a5f05a8433580f881986cf05b22"
format_stamp: "Formatted at 2026-08-07 19:45:18 on dist-test-slave-btw8"
server_key: "3df0a654fc0b0068b2e69e07d47398aa"
server_key_iv: "6edc2da3bae8912f477b95ed892c09ad"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.739337 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:18.759032 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:18.764268 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:18.764660 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:18.765839 11911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:18.767414 11911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:18.767525 11911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.767691 11911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:18.767767 11911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.808171 11911 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:33901
I20260807 19:45:18.808225 12044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:33901 every 8 connection(s)
I20260807 19:45:18.809690 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:18.813701 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 11911
I20260807 19:45:18.814013 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:45:18.814885 795 external_mini_cluster.cc:1428] Setting key 17da8c7ed6212a4298ccb42dfe59b280
Aug 07 19:45:18 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:45:18.838138 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.812779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40571 (local address 127.0.198.254:34253)
0807 19:45:18.813485 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.813495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.814547 (+ 1052us) server_negotiation.cc:407] Connection header received
0807 19:45:18.817238 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.817251 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.817383 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.817522 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.820082 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.820874 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.823731 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.824815 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.824994 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.830638 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.830648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.830656 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.830713 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.834048 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.835022 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.835031 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.835038 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.835212 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.835757 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.835765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.835771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.836212 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.836381 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.837327 (+ 946us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.837628 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":341,"threads_started":1}
I20260807 19:45:18.839143 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.813409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34253 (local address 127.0.198.193:40571)
0807 19:45:18.814310 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.814367 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.816893 (+ 2526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.817883 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.817899 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.818816 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:18.819813 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.819847 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.821084 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.821093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.823346 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.823398 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.825163 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.825174 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.825571 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.826633 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:18.826686 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:18.830247 (+ 3561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:18.834268 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.834277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.834293 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.834836 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.835331 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.835335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.835337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.835617 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.836388 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:18.836395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:18.836851 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.837156 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.837720 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":375,"threads_started":1}
I20260807 19:45:18.841154 12045 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34253
I20260807 19:45:18.841529 12045 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:18.842566 12045 heartbeater.cc:511] Master 127.0.198.254:34253 requested a full tablet report, sending...
I20260807 19:45:18.845710 11848 ts_manager.cc:194] Registered new tserver with Master: c8594a5f05a8433580f881986cf05b22 (127.0.198.193:33901)
I20260807 19:45:18.848245 11848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:40571
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:45:18.898996 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:19.272984 12052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:19.273325 12052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:19.273427 12052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:19.281057 12052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:19.281200 12052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:19.281394 12052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:45:19.291936 12052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:19.293529 12052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:19.294935 12052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:19.309643 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.309577 12058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.309620 12057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:19.316291 12052 server_base.cc:1034] running on GCE node
I20260807 19:45:19.317173 12052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:19.318403 12052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:19.319713 12052 hybrid_clock.cc:648] HybridClock initialized: now 1786131919319678 us; error 77 us; skew 500 ppm
I20260807 19:45:19.320186 12052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:19 dist-test-slave-btw8 krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:19.324349 12052 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:45:19.326577 12052 webserver.cc:460] Webserver started at http://127.0.198.194:33237/ using document root <none> and password file <none>
I20260807 19:45:19.327538 12052 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:19.327667 12052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:19.327973 12052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:19.331224 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "0f6a4d23a0e049bc9896465a9d20de80"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "9156725da04bab5690eacde4d60b5ed3"
server_key_iv: "0c0c81754e6a2062b6df0428ece496a4"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.332161 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "0f6a4d23a0e049bc9896465a9d20de80"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "9156725da04bab5690eacde4d60b5ed3"
server_key_iv: "0c0c81754e6a2062b6df0428ece496a4"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.338017 12052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:45:19.342406 12067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.344266 12052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:19.344508 12052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "0f6a4d23a0e049bc9896465a9d20de80"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "9156725da04bab5690eacde4d60b5ed3"
server_key_iv: "0c0c81754e6a2062b6df0428ece496a4"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.344779 12052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:19.376489 12052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:19.381634 12052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:19.381990 12052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:19.383068 12052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:19.385216 12052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:19.385329 12052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.385470 12052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:19.385555 12052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.422755 12052 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:37551
I20260807 19:45:19.422819 12184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:37551 every 8 connection(s)
I20260807 19:45:19.424299 12052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:19.426390 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 12052
I20260807 19:45:19.426623 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:45:19.427196 795 external_mini_cluster.cc:1428] Setting key bb7c58778a61817cbac0e7cefc2174f9
Aug 07 19:45:19 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:45:19.455986 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.427815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:54721 (local address 127.0.198.254:34253)
0807 19:45:19.428376 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:19.428383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:19.430047 (+ 1664us) server_negotiation.cc:407] Connection header received
0807 19:45:19.433547 (+ 3500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:19.433553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:19.433635 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:19.433755 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:19.436579 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.437659 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.440725 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.441439 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.441584 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:19.447191 (+ 5607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:19.447197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:19.447201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:19.447246 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:19.451467 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.452616 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.452626 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.452634 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.452882 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.453481 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.453495 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.453502 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.454023 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:19.454215 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:19.455066 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 19:45:19.455433 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":267,"threads_started":1}
I20260807 19:45:19.456739 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.428541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34253 (local address 127.0.198.194:54721)
0807 19:45:19.429845 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.429908 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:19.433130 (+ 3222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:19.434106 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:19.434123 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:19.435093 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:19.436162 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:19.436303 (+ 141us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.437896 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.437904 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:19.440524 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:19.440550 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.441737 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.441748 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:19.442140 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:19.443171 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:19.443216 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:19.446849 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:19.451830 (+ 4981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:19.451838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:19.451855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:19.452407 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:19.453082 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:19.453086 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:19.453088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:19.453325 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:19.454202 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:19.454209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:19.454594 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 19:45:19.454865 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 19:45:19.455337 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":586,"threads_started":1}
I20260807 19:45:19.458478 12185 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34253
I20260807 19:45:19.458895 12185 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:19.459955 12185 heartbeater.cc:511] Master 127.0.198.254:34253 requested a full tablet report, sending...
I20260807 19:45:19.461949 11848 ts_manager.cc:194] Registered new tserver with Master: 0f6a4d23a0e049bc9896465a9d20de80 (127.0.198.194:37551)
I20260807 19:45:19.462845 11848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:54721
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:45:19.508733 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:19.853222 12045 heartbeater.cc:503] Master 127.0.198.254:34253 was elected leader, sending a full tablet report...
W20260807 19:45:19.879158 12191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:19.879509 12191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:19.879611 12191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:19.887137 12191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:19.887279 12191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:19.887482 12191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:45:19.898041 12191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:34253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:45:19.899477 12191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:19.902052 12191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:19.919450 12197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.919507 12196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.920217 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:19.926095 12191 server_base.cc:1034] running on GCE node
I20260807 19:45:19.927016 12191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:19.928552 12191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:19.929986 12191 hybrid_clock.cc:648] HybridClock initialized: now 1786131919929887 us; error 48 us; skew 500 ppm
I20260807 19:45:19.930794 12191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:19 dist-test-slave-btw8 krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:19.935165 12191 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:45:19.937178 12191 webserver.cc:460] Webserver started at http://127.0.198.195:35675/ using document root <none> and password file <none>
I20260807 19:45:19.938007 12191 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:19.938129 12191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:19.938419 12191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:19.942070 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "ae2f05cb8b0044188d58e363c9f6cc23"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "5f25df770f9247d5a8521e887f445985"
server_key_iv: "3774405584fbac33fa74857745ac12b3"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.943287 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "ae2f05cb8b0044188d58e363c9f6cc23"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "5f25df770f9247d5a8521e887f445985"
server_key_iv: "3774405584fbac33fa74857745ac12b3"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.950954 12191 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 19:45:19.955132 12208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.956911 12191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:19.957129 12191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "ae2f05cb8b0044188d58e363c9f6cc23"
format_stamp: "Formatted at 2026-08-07 19:45:19 on dist-test-slave-btw8"
server_key: "5f25df770f9247d5a8521e887f445985"
server_key_iv: "3774405584fbac33fa74857745ac12b3"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.957438 12191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:19.988921 12191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:19.994266 12191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:19.994551 12191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:19.995708 12191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:19.997252 12191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:19.997328 12191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.997419 12191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:19.997464 12191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:20.037990 12191 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:38595
I20260807 19:45:20.038041 12321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:38595 every 8 connection(s)
I20260807 19:45:20.039516 12191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:20.047712 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 12191
I20260807 19:45:20.047938 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:20.048470 795 external_mini_cluster.cc:1428] Setting key 750ff55d25b86dff827834a2556e73af
Aug 07 19:45:20 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:45:20.064827 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.042373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:43431 (local address 127.0.198.254:34253)
0807 19:45:20.042906 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.042912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.044011 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 19:45:20.046021 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.046026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.046144 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.046247 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:20.048223 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.049407 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.051297 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.051945 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.052102 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.056387 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:20.056394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:20.056398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:20.056442 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:20.060164 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.061154 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.061159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.061162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.061319 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.061930 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.061934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.061937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.062528 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:20.062887 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.064101 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.064433 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
I20260807 19:45:20.066066 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.042919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34253 (local address 127.0.198.195:43431)
0807 19:45:20.043802 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.043859 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.045733 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.046520 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.046538 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.047109 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:20.047908 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.047944 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.049615 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.049619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.051112 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.051130 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.052199 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.052208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.052464 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.053262 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:20.053287 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:20.055960 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:20.060396 (+ 4436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:20.060406 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:20.060423 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:20.061001 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:20.061481 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:20.061485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:20.061488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:20.061757 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:20.062997 (+ 1240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:20.063005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:20.063596 (+ 591us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.063928 (+ 332us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.064550 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":405,"threads_started":1}
I20260807 19:45:20.067783 12322 heartbeater.cc:347] Connected to a master server at 127.0.198.254:34253
I20260807 19:45:20.068092 12322 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:20.068833 12322 heartbeater.cc:511] Master 127.0.198.254:34253 requested a full tablet report, sending...
I20260807 19:45:20.070443 11848 ts_manager.cc:194] Registered new tserver with Master: ae2f05cb8b0044188d58e363c9f6cc23 (127.0.198.195:38595)
I20260807 19:45:20.071285 11848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:43431
I20260807 19:45:20.078162 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:20.100085 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.083750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.0.198.254:34253)
0807 19:45:20.083983 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.083989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.084969 (+ 980us) server_negotiation.cc:407] Connection header received
0807 19:45:20.085236 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.085241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.085317 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.085442 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:20.087052 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.087895 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.089273 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.089749 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.089888 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.092814 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:20.092821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:20.092827 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:20.092878 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:20.096809 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.097765 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.097771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.097775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.097944 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.098386 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.098391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.098395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.098864 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:20.099131 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.099469 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.099736 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:45:20.105320 11848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:45:20.114655 11848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:45:20.152678 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.130221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.0.198.194:37551)
0807 19:45:20.142310 (+ 12089us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.142325 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.142375 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:45:20.142529 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.142542 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.142923 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.143219 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:20.144261 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.145685 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.149115 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.150732 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.150900 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.151123 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.152014 (+ 891us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.152239 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11789,"thread_start_us":13254,"threads_started":1}
I20260807 19:45:20.152837 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.129592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:37551 (local address 127.0.0.1:43564)
0807 19:45:20.142085 (+ 12493us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.142117 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.142342 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.143284 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.143297 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.143343 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:20.144047 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.144070 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.145897 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.145902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.148918 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.148941 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.151305 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.151313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.151563 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.151663 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.151821 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.152181 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12333,"mutex_wait_us":102,"thread_start_us":12177,"threads_started":1}
I20260807 19:45:20.152989 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.129294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:38595 (local address 127.0.0.1:55798)
0807 19:45:20.136645 (+ 7351us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.136692 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.136924 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.145207 (+ 8283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.145212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.145244 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:20.145739 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.145756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.147829 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.147838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.149840 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.149853 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.151723 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.151729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.151924 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.151960 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.152051 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.152197 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7060,"mutex_wait_us":4,"thread_start_us":6850,"threads_started":1}
I20260807 19:45:20.153904 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.129698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.0.198.195:38595)
0807 19:45:20.144403 (+ 14705us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.144415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.144457 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:45:20.144573 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.144585 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.144961 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.145215 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:20.146014 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.147655 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.150050 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.151495 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.151653 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.151884 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.153323 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.153526 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14460,"thread_start_us":14245,"threads_started":1}
I20260807 19:45:20.157029 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.143546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:33901 (local address 127.0.0.1:55404)
0807 19:45:20.146049 (+ 2503us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.146067 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.146229 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.147673 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.147677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.147704 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:20.148244 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.148259 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.151341 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.151372 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.153620 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.153641 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.156135 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.156142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.156346 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.156385 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.156497 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.156657 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2370,"thread_start_us":2194,"threads_started":1}
I20260807 19:45:20.157158 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.137732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.0.198.193:33901)
0807 19:45:20.146968 (+ 9236us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.146977 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.147010 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:45:20.147112 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.147119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.147424 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.148208 (+ 784us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:20.148675 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.151174 (+ 2499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.154197 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.155872 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.156037 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.156278 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.156564 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.156804 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9041,"thread_start_us":9244,"threads_started":1}
I20260807 19:45:20.157626 12256 tablet_service.cc:1467] Processing CreateTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 (DEFAULT_TABLE table=test-table [id=6e071b9279bd42f2abbbe49e8739de9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:20.157608 12118 tablet_service.cc:1467] Processing CreateTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 (DEFAULT_TABLE table=test-table [id=6e071b9279bd42f2abbbe49e8739de9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:20.159153 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 065e6b2152ec47f5873fe04c1f0bf710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:20.159158 12118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 065e6b2152ec47f5873fe04c1f0bf710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:20.161518 11977 tablet_service.cc:1467] Processing CreateTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 (DEFAULT_TABLE table=test-table [id=6e071b9279bd42f2abbbe49e8739de9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:20.163389 11977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 065e6b2152ec47f5873fe04c1f0bf710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:20.174741 12342 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Bootstrap starting.
I20260807 19:45:20.176308 12343 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Bootstrap starting.
I20260807 19:45:20.178921 12342 tablet_bootstrap.cc:654] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:20.179819 12344 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Bootstrap starting.
I20260807 19:45:20.180661 12343 tablet_bootstrap.cc:654] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:20.181062 12342 log.cc:824] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:20.182443 12343 log.cc:824] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:20.183682 12344 tablet_bootstrap.cc:654] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:20.185653 12344 log.cc:824] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:20.186484 12343 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: No bootstrap required, opened a new log
I20260807 19:45:20.186828 12343 ts_tablet_manager.cc:1397] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260807 19:45:20.187932 12342 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: No bootstrap required, opened a new log
I20260807 19:45:20.188252 12342 ts_tablet_manager.cc:1397] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.009s
I20260807 19:45:20.188835 12344 tablet_bootstrap.cc:492] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: No bootstrap required, opened a new log
I20260807 19:45:20.189131 12344 ts_tablet_manager.cc:1397] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:45:20.198414 12344 raft_consensus.cc:348] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.198525 12343 raft_consensus.cc:348] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.198904 12344 raft_consensus.cc:374] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:20.198995 12344 raft_consensus.cc:729] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8594a5f05a8433580f881986cf05b22, State: Initialized, Role: FOLLOWER
I20260807 19:45:20.199009 12343 raft_consensus.cc:374] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:20.199115 12343 raft_consensus.cc:729] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2f05cb8b0044188d58e363c9f6cc23, State: Initialized, Role: FOLLOWER
I20260807 19:45:20.199752 12344 consensus_queue.cc:260] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.199666 12342 raft_consensus.cc:348] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.199901 12343 consensus_queue.cc:260] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.200155 12342 raft_consensus.cc:374] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:20.200275 12342 raft_consensus.cc:729] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f6a4d23a0e049bc9896465a9d20de80, State: Initialized, Role: FOLLOWER
I20260807 19:45:20.201159 12342 consensus_queue.cc:260] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.201833 12344 ts_tablet_manager.cc:1428] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260807 19:45:20.204010 12322 heartbeater.cc:503] Master 127.0.198.254:34253 was elected leader, sending a full tablet report...
I20260807 19:45:20.205040 12343 ts_tablet_manager.cc:1428] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260807 19:45:20.206148 12185 heartbeater.cc:503] Master 127.0.198.254:34253 was elected leader, sending a full tablet report...
I20260807 19:45:20.206872 12342 ts_tablet_manager.cc:1428] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Time spent starting tablet: real 0.018s user 0.012s sys 0.006s
W20260807 19:45:20.291527 12323 tablet.cc:2369] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:45:20.312783 12046 tablet.cc:2369] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:45:20.426828 12186 tablet.cc:2369] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:20.578562 12348 raft_consensus.cc:482] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:20.578997 12348 raft_consensus.cc:504] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.580957 12348 leader_election.cc:288] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae2f05cb8b0044188d58e363c9f6cc23 (127.0.198.195:38595), 0f6a4d23a0e049bc9896465a9d20de80 (127.0.198.194:37551)
I20260807 19:45:20.588413 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.581328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40073 (local address 127.0.198.195:38595)
0807 19:45:20.581592 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.581602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.582236 (+ 634us) server_negotiation.cc:407] Connection header received
0807 19:45:20.582483 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.582490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.582584 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.582685 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:20.583489 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.584253 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.586244 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.587240 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.587452 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.587513 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.587956 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.588105 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:45:20.588456 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.581829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:41699 (local address 127.0.198.194:37551)
0807 19:45:20.582089 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.582097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.582603 (+ 506us) server_negotiation.cc:407] Connection header received
0807 19:45:20.582784 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.582789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.582886 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.582995 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:20.583657 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.584467 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.586493 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.587469 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.587657 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.587732 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.588047 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.588188 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:45:20.588562 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.581613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:37551 (local address 127.0.198.193:41699)
0807 19:45:20.582472 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.582490 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.582642 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.583017 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.583024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.583044 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:20.583468 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.583480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.584737 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.584741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.586234 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.586248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.587704 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.587709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.587832 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.587855 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.587980 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.588111 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":5,"thread_start_us":456,"threads_started":1}
I20260807 19:45:20.588969 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.581187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:38595 (local address 127.0.198.193:40073)
0807 19:45:20.582089 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.582117 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:20.582312 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:20.582767 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:20.582780 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:20.582825 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:20.583287 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.583303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.584410 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.584418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.586040 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:20.586053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.587518 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.587528 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:20.587764 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.587805 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:45:20.587889 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:45:20.588056 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":442,"threads_started":1}
I20260807 19:45:20.589679 12139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "065e6b2152ec47f5873fe04c1f0bf710" candidate_uuid: "c8594a5f05a8433580f881986cf05b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6a4d23a0e049bc9896465a9d20de80" is_pre_election: true
I20260807 19:45:20.590004 12277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "065e6b2152ec47f5873fe04c1f0bf710" candidate_uuid: "c8594a5f05a8433580f881986cf05b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" is_pre_election: true
I20260807 19:45:20.590466 12277 raft_consensus.cc:2454] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8594a5f05a8433580f881986cf05b22 in term 0.
I20260807 19:45:20.590466 12139 raft_consensus.cc:2454] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8594a5f05a8433580f881986cf05b22 in term 0.
I20260807 19:45:20.591344 11930 leader_election.cc:302] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6a4d23a0e049bc9896465a9d20de80, c8594a5f05a8433580f881986cf05b22; no voters:
I20260807 19:45:20.592006 12348 raft_consensus.cc:2781] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:20.592190 12348 raft_consensus.cc:482] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:20.592307 12348 raft_consensus.cc:3037] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:20.594677 12348 raft_consensus.cc:504] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.595495 12348 leader_election.cc:288] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [CANDIDATE]: Term 1 election: Requested vote from peers ae2f05cb8b0044188d58e363c9f6cc23 (127.0.198.195:38595), 0f6a4d23a0e049bc9896465a9d20de80 (127.0.198.194:37551)
I20260807 19:45:20.596148 12139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "065e6b2152ec47f5873fe04c1f0bf710" candidate_uuid: "c8594a5f05a8433580f881986cf05b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6a4d23a0e049bc9896465a9d20de80"
I20260807 19:45:20.596165 12277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "065e6b2152ec47f5873fe04c1f0bf710" candidate_uuid: "c8594a5f05a8433580f881986cf05b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2f05cb8b0044188d58e363c9f6cc23"
I20260807 19:45:20.596431 12277 raft_consensus.cc:3037] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:20.596412 12139 raft_consensus.cc:3037] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:20.598834 12139 raft_consensus.cc:2454] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8594a5f05a8433580f881986cf05b22 in term 1.
I20260807 19:45:20.598958 12277 raft_consensus.cc:2454] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8594a5f05a8433580f881986cf05b22 in term 1.
I20260807 19:45:20.599568 11930 leader_election.cc:302] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6a4d23a0e049bc9896465a9d20de80, c8594a5f05a8433580f881986cf05b22; no voters:
I20260807 19:45:20.599951 12348 raft_consensus.cc:2781] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:20.600533 12348 raft_consensus.cc:686] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 1 LEADER]: Becoming Leader. State: Replica: c8594a5f05a8433580f881986cf05b22, State: Running, Role: LEADER
I20260807 19:45:20.601110 12348 consensus_queue.cc:237] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } }
I20260807 19:45:20.606228 11848 catalog_manager.cc:5495] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8594a5f05a8433580f881986cf05b22 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "c8594a5f05a8433580f881986cf05b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8594a5f05a8433580f881986cf05b22" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 33901 } health_report { overall_health: HEALTHY } } }
I20260807 19:45:20.630527 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.623434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.0.198.193:33901)
0807 19:45:20.623780 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.623788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.624015 (+ 227us) server_negotiation.cc:407] Connection header received
0807 19:45:20.624329 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.624333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.624447 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.624660 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:20.625189 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.626011 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.627722 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.628201 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.628365 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.629008 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:20.629772 (+ 764us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:20.629966 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.630122 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.630227 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:45:20.640698 12139 raft_consensus.cc:1261] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 1 FOLLOWER]: Refusing update from remote peer c8594a5f05a8433580f881986cf05b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:20.640719 12277 raft_consensus.cc:1261] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 1 FOLLOWER]: Refusing update from remote peer c8594a5f05a8433580f881986cf05b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:20.641785 12353 consensus_queue.cc:1035] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f6a4d23a0e049bc9896465a9d20de80" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:20.642216 12348 consensus_queue.cc:1035] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae2f05cb8b0044188d58e363c9f6cc23" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:20.658604 12358 mvcc.cc:205] Tried to move back new op lower bound from 7315996346932146176 to 7315996346785914880. Current Snapshot: MvccSnapshot[applied={T|T < 7315996346932146176}]
I20260807 19:45:20.673365 11848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:20.673687 11848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:20.677748 11848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63406ccb3c014d93b25ca853a6ebd37e: Sending DeleteTablet for 3 replicas of tablet 065e6b2152ec47f5873fe04c1f0bf710
I20260807 19:45:20.679224 11977 tablet_service.cc:1514] Processing DeleteTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:20 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:55404
I20260807 19:45:20.679955 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:20 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:55798
I20260807 19:45:20.679159 12118 tablet_service.cc:1514] Processing DeleteTablet for tablet 065e6b2152ec47f5873fe04c1f0bf710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:20 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:43564
I20260807 19:45:20.680670 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11911
I20260807 19:45:20.682109 12368 tablet_replica.cc:330] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: stopping tablet replica
I20260807 19:45:20.682782 12368 raft_consensus.cc:2229] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:45:20.684032 12368 raft_consensus.cc:2258] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:20.689311 12368 ts_tablet_manager.cc:1905] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:20.695899 12369 tablet_replica.cc:330] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: stopping tablet replica
I20260807 19:45:20.696442 12369 raft_consensus.cc:2229] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:20.697036 12369 raft_consensus.cc:2258] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:20.698614 12368 ts_tablet_manager.cc:1918] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:20.698827 12368 log.cc:1196] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/065e6b2152ec47f5873fe04c1f0bf710
I20260807 19:45:20.699318 12369 ts_tablet_manager.cc:1905] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:20.699565 12370 tablet_replica.cc:330] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: stopping tablet replica
I20260807 19:45:20.700052 12370 raft_consensus.cc:2229] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:20.700608 12370 raft_consensus.cc:2258] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:20.701107 12368 ts_tablet_manager.cc:1939] T 065e6b2152ec47f5873fe04c1f0bf710 P c8594a5f05a8433580f881986cf05b22: Deleting consensus metadata
I20260807 19:45:20.702689 12370 ts_tablet_manager.cc:1905] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:45:20 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:45:20.707530 12369 ts_tablet_manager.cc:1918] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:20.707754 12369 log.cc:1196] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/065e6b2152ec47f5873fe04c1f0bf710
I20260807 19:45:20.708288 12369 ts_tablet_manager.cc:1939] T 065e6b2152ec47f5873fe04c1f0bf710 P ae2f05cb8b0044188d58e363c9f6cc23: Deleting consensus metadata
I20260807 19:45:20.710383 11833 catalog_manager.cc:4841] TS ae2f05cb8b0044188d58e363c9f6cc23 (127.0.198.195:38595): tablet 065e6b2152ec47f5873fe04c1f0bf710 (table test-table [id=6e071b9279bd42f2abbbe49e8739de9b]) successfully deleted
I20260807 19:45:20.711311 12370 ts_tablet_manager.cc:1918] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:20.711544 12370 log.cc:1196] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/065e6b2152ec47f5873fe04c1f0bf710
I20260807 19:45:20.712102 12370 ts_tablet_manager.cc:1939] T 065e6b2152ec47f5873fe04c1f0bf710 P 0f6a4d23a0e049bc9896465a9d20de80: Deleting consensus metadata
I20260807 19:45:20.712343 11833 catalog_manager.cc:4841] TS c8594a5f05a8433580f881986cf05b22 (127.0.198.193:33901): tablet 065e6b2152ec47f5873fe04c1f0bf710 (table test-table [id=6e071b9279bd42f2abbbe49e8739de9b]) successfully deleted
I20260807 19:45:20.714020 11834 catalog_manager.cc:4841] TS 0f6a4d23a0e049bc9896465a9d20de80 (127.0.198.194:37551): tablet 065e6b2152ec47f5873fe04c1f0bf710 (table test-table [id=6e071b9279bd42f2abbbe49e8739de9b]) successfully deleted
I20260807 19:45:20.717693 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.682894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.0.198.193:33901)
0807 19:45:20.683142 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.683152 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.692760 (+ 9608us) server_negotiation.cc:407] Connection header received
0807 19:45:20.693307 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.693313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.693408 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.693478 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:20.696284 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.697587 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.700043 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.700812 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.700924 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.707768 (+ 6844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:20.707790 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:20.707812 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:20.707871 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:20.714021 (+ 6150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.715241 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.715250 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.715252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.715525 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.716017 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.716021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.716023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.716416 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:20.716675 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.717090 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.717319 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:45:20.718487 12040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:20.821300 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11911
I20260807 19:45:20.840509 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 12052
Aug 07 19:45:20 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:45:20.857426 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.841315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.0.198.194:37551)
0807 19:45:20.841517 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.841525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.841552 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:45:20.841864 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.841869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.841939 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.842083 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:20.843814 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.844581 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.845852 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.846277 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.846427 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.851513 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:20.851531 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:20.851548 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:20.851590 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:20.854494 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.855289 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.855297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.855299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.855528 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.855907 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.855910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.855912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.856302 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:20.856548 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.856844 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.857020 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:45:20.858230 12180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:20.951457 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 12052
I20260807 19:45:20.974366 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 12191
Aug 07 19:45:20 dist-test-slave-btw8 krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:45:20.991974 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.975307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.0.198.195:38595)
0807 19:45:20.975543 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:20.975551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:20.975608 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:45:20.975835 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:20.975841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:20.975918 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:20.976026 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:20.977312 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.978072 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.979532 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:20.979995 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:20.980129 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:20.984913 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:20.984933 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:20.984951 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:20.985007 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:20.988344 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.989288 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.989297 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.989300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.989481 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:20.990025 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:20.990030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:20.990032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:20.990494 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:20.990770 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:20.991212 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 19:45:20.991527 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 19:45:20.992938 12317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:21.086220 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 12191
I20260807 19:45:21.111910 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 11816
I20260807 19:45:21.113356 11879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:21.230247 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 11816
2026-08-07T19:45:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5196 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:21 dist-test-slave-btw8 krb5kdc[12379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:21 dist-test-slave-btw8 krb5kdc[12379](info): set up 2 sockets
Aug 07 19:45:21 dist-test-slave-btw8 krb5kdc[12379](info): commencing operation
krb5kdc: starting...
W20260807 19:45:22.442158 795 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:45:22 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangeradmin_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangeradmin_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangerlookup_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangerlookup_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
I20260807 19:45:22.951488 795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYvfPwA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/postgres -l logfile start
2026-08-07 19:45:25.988 UTC [12410] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:45:25.988 UTC [12410] LOG: listening on IPv4 address "127.0.198.212", port 47613
2026-08-07 19:45:25.994 UTC [12410] LOG: listening on Unix socket "/tmp/.s.PGSQL.47613"
2026-08-07 19:45:26.020 UTC [12412] LOG: database system was shut down at 2026-08-07 19:45:24 UTC
2026-08-07 19:45:26.025 UTC [12410] LOG: database system is ready to accept connections
I20260807 19:45:27.110198 795 mini_postgres.cc:96] Postgres bound to 47613
2026-08-07 19:45:27.140 UTC [12421] FATAL: database "slave" does not exist
127.0.198.212:47613 - accepting connections
I20260807 19:45:27.141414 795 mini_ranger.cc:161] Starting Ranger...
I20260807 19:45:27.182168 795 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:45:27.826400 795 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:45:27.826565 795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin
2026-08-07 19:45:28,233 [I] DB FLAVOR :POSTGRES
2026-08-07 19:45:28,234 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:45:28,234 [I] Checking connection..
2026-08-07 19:45:28,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:45:28,541 [I] Checking connection passed.
2026-08-07 19:45:28,541 [I] --------- Verifying version history table ---------
2026-08-07 19:45:28,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:45:28,800 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:45:28,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:45:29,094 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:45:29,094 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:45:29,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:45:29.364 UTC [12487] WARNING: there is no transaction in progress
2026-08-07 19:45:29,372 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:45:29,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:45:29,643 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:45:29,643 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:45:29,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:45:29,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:45:30,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-btw8.c.gcp-upstream.internal','N') ;"
2026-08-07 19:45:30,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:45:30,745 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:45:30,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:45:31,026 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:45:31,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:45:31,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:45:31,533 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:45:31,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:45:32.677 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.696 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.717 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.736 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.757 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.780 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.849 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.857 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.873 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.885 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.896 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.913 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.926 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.935 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.945 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:32.955 UTC [12598] WARNING: there is no transaction in progress
2026-08-07 19:45:33,827 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:45:33,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:45:34,116 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:45:34,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:45:34,396 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:45:34,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:45:34,670 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:45:34,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:45:34,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:45:35,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-btw8.c.gcp-upstream.internal';"
2026-08-07 19:45:35,496 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:45:35,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:45:35,762 [I] DB_PATCHES have already been applied
I20260807 19:45:35.769511 795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYvfPwA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:45:35.769618 795 mini_ranger.cc:193] Using host: 127.0.198.212
I20260807 19:45:35.795725 795 mini_ranger.cc:227] Ranger admin URL: http://127.0.198.212:43681
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:45:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:45:37.077742 795 mini_ranger.cc:160] Time spent starting Ranger: real 9.936s user 0.000s sys 0.084s
Aug 07, 2026 7:45:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:45:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 7:45:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:45:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:46:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:46:01.276010 795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangerkms_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/rangerkms_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.198.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.198.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:46:01 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131961, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:46:01.554483 795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:46:01.587816 795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:46:02.771299 795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:46:02.771530 795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms
I20260807 19:46:02.771677 795 mini_ranger_kms.cc:227] Using postgres at 127.0.198.212:47613
2026-08-07 19:46:03,714 [I] DB FLAVOR :POSTGRES
2026-08-07 19:46:03,714 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:46:03,715 [I] Checking connection
2026-08-07 19:46:03,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:46:03,991 [I] connection success
2026-08-07 19:46:03,991 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:46:03,991 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:46:03,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:46:04,252 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:46:04,253 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:46:04,253 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:46:04,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:47613/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:46:04,537 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:46:05.136458 795 mini_ranger_kms.cc:310] Created kms service
I20260807 19:46:05.343547 795 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:46:05.393776 795 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:46:05 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131961, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:05.846109 795 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:46:05.847939 795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:46:05.848052 795 mini_ranger_kms.cc:240] Using host: 127.0.198.212
I20260807 19:46:05.887085 795 mini_ranger_kms.cc:276] Ranger KMS PID: 12792
I20260807 19:46:05.887674 795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.198.212:45683
Aug 07, 2026 7:46:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:46:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 19:46:08.334022 795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.780s user 0.143s sys 0.536s
I20260807 19:46:08.334398 795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:46:08.334640 795 mini_ranger_kms.cc:401] 127.0.198.212:45683/kms/v1/keys
Aug 07 19:46:08 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131968, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 19:46:08 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131968, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
Aug 07 19:46:09 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131968, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
Aug 07 19:46:11 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:46:11.849098 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40399
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:40399
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:45683/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:12.240144 12843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:12.240490 12843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:12.240650 12843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:12.248407 12843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:12.248538 12843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:12.248638 12843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:12.248740 12843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:12.248833 12843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:12.260174 12843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.198.212:45683/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:40399
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:46:12.261874 12843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:12.263370 12843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:12.276904 12849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:12.276939 12848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:12.277032 12851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:12.277669 12843 server_base.cc:1034] running on GCE node
I20260807 19:46:12.278743 12843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:12.280447 12843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:12.281698 12843 hybrid_clock.cc:648] HybridClock initialized: now 1786131972281662 us; error 51 us; skew 500 ppm
I20260807 19:46:12.282086 12843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:12 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131972, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:12.288831 12843 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:46:12.290606 12843 webserver.cc:460] Webserver started at http://127.0.198.254:33783/ using document root <none> and password file <none>
I20260807 19:46:12.291409 12843 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:12.291558 12843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:12.291893 12843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:46:12 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131972, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:12.348593 12843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "56889a1cb12146c9898c5002a03e7dd7"
format_stamp: "Formatted at 2026-08-07 19:46:12 on dist-test-slave-btw8"
server_key: "0e5fdee86eb94c141f0b1d6ef9274a5f"
server_key_iv: "4def03ecd9d042f400411f73354b6d9e"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:12.349592 12843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "56889a1cb12146c9898c5002a03e7dd7"
format_stamp: "Formatted at 2026-08-07 19:46:12 on dist-test-slave-btw8"
server_key: "0e5fdee86eb94c141f0b1d6ef9274a5f"
server_key_iv: "4def03ecd9d042f400411f73354b6d9e"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:12.356861 12843 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 19:46:12.375558 12858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:12.377203 12843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:46:12.377441 12843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "56889a1cb12146c9898c5002a03e7dd7"
format_stamp: "Formatted at 2026-08-07 19:46:12 on dist-test-slave-btw8"
server_key: "0e5fdee86eb94c141f0b1d6ef9274a5f"
server_key_iv: "4def03ecd9d042f400411f73354b6d9e"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:12.377727 12843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:12.395022 12843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:12.400189 12843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:12.400605 12843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:12.425171 12843 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:40399
I20260807 19:46:12.425210 12910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:40399 every 8 connection(s)
I20260807 19:46:12.426810 12843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:12.431525 12911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:12.433643 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 12843
I20260807 19:46:12.433917 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:46:12.434970 795 external_mini_cluster.cc:1428] Setting key 2475f4c24493663e35213744d30d6075
I20260807 19:46:12.447127 12911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: Bootstrap starting.
Aug 07 19:46:12 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:46:12.451897 12911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:12.453442 12911 log.cc:824] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:12.457607 12911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: No bootstrap required, opened a new log
I20260807 19:46:12.461944 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:12.437466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.0.198.254:40399)
0807 19:46:12.438437 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:12.438450 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:12.438499 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:46:12.440678 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:12.440726 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:12.441325 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:12.441671 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:12.443667 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:12.445041 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:12.446977 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:12.447797 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:12.447928 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:12.453378 (+ 5450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:12.453387 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:12.453402 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:12.453452 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:12.457346 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:12.458322 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:12.458337 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:12.458345 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:12.458514 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:12.458920 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:12.458927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:12.458935 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:12.459543 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:12.459890 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:12.460382 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 19:46:12.460820 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":486,"threads_started":1}
I20260807 19:46:12.467159 12911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } }
I20260807 19:46:12.467769 12911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:12.467900 12911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56889a1cb12146c9898c5002a03e7dd7, State: Initialized, Role: FOLLOWER
I20260807 19:46:12.468626 12911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } }
I20260807 19:46:12.468964 12911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:12.469179 12911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:12.469444 12911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:12.472597 12911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } }
I20260807 19:46:12.473404 12911 leader_election.cc:302] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56889a1cb12146c9898c5002a03e7dd7; no voters:
I20260807 19:46:12.474031 12911 leader_election.cc:288] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:12.474200 12916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:12.476080 12916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [term 1 LEADER]: Becoming Leader. State: Replica: 56889a1cb12146c9898c5002a03e7dd7, State: Running, Role: LEADER
I20260807 19:46:12.476640 12916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } }
I20260807 19:46:12.476810 12911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:12.480389 12918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56889a1cb12146c9898c5002a03e7dd7. Latest consensus state: current_term: 1 leader_uuid: "56889a1cb12146c9898c5002a03e7dd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } } }
I20260807 19:46:12.480944 12918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:12.480827 12917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56889a1cb12146c9898c5002a03e7dd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56889a1cb12146c9898c5002a03e7dd7" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40399 } } }
I20260807 19:46:12.481182 12917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7 [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:12.482808 12924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:12.488588 12924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:12.490101 12911 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:46:12.510842 12924 catalog_manager.cc:1347] Generated new cluster ID: d6556a4ae5ff4dcf94f38aef76e8d258
I20260807 19:46:12.511011 12924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:12.544683 12924 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:12.546346 12924 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:12.556748 12924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: Generated new TSK 0
I20260807 19:46:12.557827 12924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:46:13.468952 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:45683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:46:13 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:46:13.930668 12958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:13.931195 12958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:13.931492 12958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:13.942873 12958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:13.943017 12958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:13.943231 12958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:46:13.961015 12958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.198.212:45683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:46:13.962997 12958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:13.965139 12958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:13.986832 12968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:13.987668 12970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:13.994351 12958 server_base.cc:1034] running on GCE node
W20260807 19:46:13.994702 12972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:13.995755 12958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:13.997156 12958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:13.998494 12958 hybrid_clock.cc:648] HybridClock initialized: now 1786131973998412 us; error 71 us; skew 500 ppm
I20260807 19:46:13.998961 12958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:13 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:14.003979 12958 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:46:14.006633 12958 webserver.cc:460] Webserver started at http://127.0.198.193:40631/ using document root <none> and password file <none>
I20260807 19:46:14.007970 12958 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:14.008188 12958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:14.008682 12958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:46:14 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:14.037369 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "90f6779e571c4bbb98b213115f3b7f51"
format_stamp: "Formatted at 2026-08-07 19:46:14 on dist-test-slave-btw8"
server_key: "5a239da10ec7d62d22b5f26decbc97f0"
server_key_iv: "b51158a954ccdfef322cd9198272c8c8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:14.038786 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "90f6779e571c4bbb98b213115f3b7f51"
format_stamp: "Formatted at 2026-08-07 19:46:14 on dist-test-slave-btw8"
server_key: "5a239da10ec7d62d22b5f26decbc97f0"
server_key_iv: "b51158a954ccdfef322cd9198272c8c8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:14.060002 12958 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.012s sys 0.006s
I20260807 19:46:14.088333 12979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:14.090932 12958 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.004s
I20260807 19:46:14.091204 12958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "90f6779e571c4bbb98b213115f3b7f51"
format_stamp: "Formatted at 2026-08-07 19:46:14 on dist-test-slave-btw8"
server_key: "5a239da10ec7d62d22b5f26decbc97f0"
server_key_iv: "b51158a954ccdfef322cd9198272c8c8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:14.091598 12958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:14.120223 12958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:14.133308 12958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:14.133801 12958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:14.155908 12958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:14.175297 12958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:14.175582 12958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:46:14.175843 12958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:14.175999 12958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:46:14 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:14.685460 12958 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:37213
I20260807 19:46:14.685541 13099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:37213 every 8 connection(s)
I20260807 19:46:14.687279 12958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:14.694607 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 12958
I20260807 19:46:14.694892 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:14.696220 795 external_mini_cluster.cc:1428] Setting key 7009b78b24edfc07089fd847c696bdda
Aug 07 19:46:14 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:46:14.756609 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:14.692107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:58359 (local address 127.0.198.254:40399)
0807 19:46:14.720091 (+ 27984us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:14.720102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:14.732340 (+ 12238us) server_negotiation.cc:407] Connection header received
0807 19:46:14.735232 (+ 2892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:14.735240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:14.735345 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:14.735456 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:14.738022 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:14.739271 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:14.741910 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:14.742599 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:14.742707 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:14.748086 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:14.748093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:14.748098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:14.748157 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:14.752155 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:14.753415 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:14.753422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:14.753426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:14.753614 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:14.754246 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:14.754251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:14.754255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:14.754638 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:14.754771 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:14.755910 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0807 19:46:14.756189 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27770,"thread_start_us":27577,"threads_started":1}
I20260807 19:46:14.759334 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:14.691309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40399 (local address 127.0.198.193:58359)
0807 19:46:14.732100 (+ 40791us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:14.732171 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:14.734789 (+ 2618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:14.736089 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:14.736102 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:14.736816 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:14.737746 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:14.737776 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:14.739560 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:14.739566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:14.741697 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:14.741715 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:14.743112 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:14.743119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:14.743481 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:14.744633 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:14.744666 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:14.747685 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:14.752472 (+ 4787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:14.752484 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:14.752508 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:14.753232 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:14.753781 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:14.753785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:14.753790 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:14.754092 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:14.754902 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:14.754912 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:14.755445 (+ 533us) client_negotiation.cc:769] Sending connection context
0807 19:46:14.757200 (+ 1755us) client_negotiation.cc:240] Negotiation successful
0807 19:46:14.757846 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40373,"thread_start_us":40126,"threads_started":1}
I20260807 19:46:14.762112 13100 heartbeater.cc:347] Connected to a master server at 127.0.198.254:40399
I20260807 19:46:14.762516 13100 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:14.763639 13100 heartbeater.cc:511] Master 127.0.198.254:40399 requested a full tablet report, sending...
I20260807 19:46:14.766776 12876 ts_manager.cc:194] Registered new tserver with Master: 90f6779e571c4bbb98b213115f3b7f51 (127.0.198.193:37213)
I20260807 19:46:14.769301 12876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:58359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
W20260807 19:46:14.812105 795 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.198.194: real 0.112s user 0.006s sys 0.026s
I20260807 19:46:14.813066 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:45683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:46:14 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
W20260807 19:46:15.345500 13106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:15.346144 13106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:15.346400 13106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:15.356233 13106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:15.356377 13106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:15.356559 13106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:46:15.367265 13106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.198.212:45683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:46:15.368894 13106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:15.370362 13106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:15.388998 13123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:15.389173 13122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:15.389397 13125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:15.390198 13106 server_base.cc:1034] running on GCE node
I20260807 19:46:15.391054 13106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:15.392406 13106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:15.393711 13106 hybrid_clock.cc:648] HybridClock initialized: now 1786131975393658 us; error 41 us; skew 500 ppm
I20260807 19:46:15.394387 13106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:15 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131975, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:15.398631 13106 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:46:15.400662 13106 webserver.cc:460] Webserver started at http://127.0.198.194:46867/ using document root <none> and password file <none>
I20260807 19:46:15.401474 13106 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:15.401648 13106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:15.402016 13106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:46:15 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131975, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:15.422650 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "c8a33c0c54814a848efbc179f43da1fa"
format_stamp: "Formatted at 2026-08-07 19:46:15 on dist-test-slave-btw8"
server_key: "59b1b694b144228c6d66cf71cd9039e9"
server_key_iv: "9bf2d371987f776308eb7e09961a8b59"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:15.424139 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "c8a33c0c54814a848efbc179f43da1fa"
format_stamp: "Formatted at 2026-08-07 19:46:15 on dist-test-slave-btw8"
server_key: "59b1b694b144228c6d66cf71cd9039e9"
server_key_iv: "9bf2d371987f776308eb7e09961a8b59"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:15.433034 13106 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260807 19:46:15.454041 13132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:15.456102 13106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:46:15.456426 13106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "c8a33c0c54814a848efbc179f43da1fa"
format_stamp: "Formatted at 2026-08-07 19:46:15 on dist-test-slave-btw8"
server_key: "59b1b694b144228c6d66cf71cd9039e9"
server_key_iv: "9bf2d371987f776308eb7e09961a8b59"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:15.456704 13106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:15.513042 13106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:15.517966 13106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:15.518335 13106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:15.519615 13106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:15.521253 13106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:15.521373 13106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:15.521546 13106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:15.521631 13106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:15.563004 13106 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:34005
I20260807 19:46:15.563020 13247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:34005 every 8 connection(s)
I20260807 19:46:15.564561 13106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:15.568946 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 13106
I20260807 19:46:15.569156 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:15.569694 795 external_mini_cluster.cc:1428] Setting key 739b9cbe9b6e08a6474ce55be7ba13c3
Aug 07 19:46:15 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131975, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:46:15.591385 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:15.567526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:35751 (local address 127.0.198.254:40399)
0807 19:46:15.568111 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:15.568117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:15.568999 (+ 882us) server_negotiation.cc:407] Connection header received
0807 19:46:15.571497 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:15.571502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:15.571577 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:15.571683 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:15.573356 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:15.574748 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:15.576776 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:15.577658 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:15.577832 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:15.582983 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:15.582993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:15.583002 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:15.583068 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:15.587180 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:15.588196 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:15.588206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:15.588212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:15.588398 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:15.588921 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:15.588930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:15.588937 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:15.589392 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:15.589572 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:15.590455 (+ 883us) server_negotiation.cc:299] Negotiation successful
0807 19:46:15.590767 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":322,"threads_started":1}
I20260807 19:46:15.591990 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:15.568017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40399 (local address 127.0.198.194:35751)
0807 19:46:15.568799 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:15.568851 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:15.570982 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:15.571989 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:15.571998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:15.572489 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:15.573161 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:15.573180 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:15.574983 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:15.574988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:15.576604 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:15.576626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:15.577930 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:15.577945 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:15.578307 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:15.579240 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:15.579273 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:15.582603 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:15.587426 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:15.587434 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:15.587450 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:15.587985 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:15.588559 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:15.588563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:15.588566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:15.588781 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:15.589580 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:15.589587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:15.589990 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 19:46:15.590252 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 19:46:15.590712 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":333,"threads_started":1}
I20260807 19:46:15.593530 13248 heartbeater.cc:347] Connected to a master server at 127.0.198.254:40399
I20260807 19:46:15.593879 13248 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:15.594626 13248 heartbeater.cc:511] Master 127.0.198.254:40399 requested a full tablet report, sending...
I20260807 19:46:15.596272 12876 ts_manager.cc:194] Registered new tserver with Master: c8a33c0c54814a848efbc179f43da1fa (127.0.198.194:34005)
I20260807 19:46:15.597656 12876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:35751
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:46:15.648698 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:45683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:46:15.774416 13100 heartbeater.cc:503] Master 127.0.198.254:40399 was elected leader, sending a full tablet report...
W20260807 19:46:16.025359 13254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:16.025789 13254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:16.025974 13254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:16.034539 13254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:16.034761 13254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:16.034981 13254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:46:16.046577 13254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.198.212:45683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:46:16.048233 13254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:16.049654 13254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:16.064957 13260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:16.064906 13259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:16.065131 13262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:16.066097 13254 server_base.cc:1034] running on GCE node
I20260807 19:46:16.066869 13254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:16.068174 13254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:16.069439 13254 hybrid_clock.cc:648] HybridClock initialized: now 1786131976069363 us; error 65 us; skew 500 ppm
I20260807 19:46:16.069892 13254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:16.073868 13254 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:46:16.075816 13254 webserver.cc:460] Webserver started at http://127.0.198.195:45401/ using document root <none> and password file <none>
I20260807 19:46:16.076591 13254 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:16.076743 13254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:16.077135 13254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:46:16.094133 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "0ec34c295e224d03b257615cc522ddbd"
format_stamp: "Formatted at 2026-08-07 19:46:16 on dist-test-slave-btw8"
server_key: "8d9207a381a2043479823236738dd3ee"
server_key_iv: "93c76b3cc70adfbffad8660051df3fb8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:16.095228 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "0ec34c295e224d03b257615cc522ddbd"
format_stamp: "Formatted at 2026-08-07 19:46:16 on dist-test-slave-btw8"
server_key: "8d9207a381a2043479823236738dd3ee"
server_key_iv: "93c76b3cc70adfbffad8660051df3fb8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:16.102278 13254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:46:16.117914 13269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:16.119668 13254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:46:16.119908 13254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "0ec34c295e224d03b257615cc522ddbd"
format_stamp: "Formatted at 2026-08-07 19:46:16 on dist-test-slave-btw8"
server_key: "8d9207a381a2043479823236738dd3ee"
server_key_iv: "93c76b3cc70adfbffad8660051df3fb8"
server_key_version: "kuduclusterkey@0"
I20260807 19:46:16.120128 13254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:16.151175 13254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:16.156790 13254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:16.157184 13254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:16.158294 13254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:16.159904 13254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:16.159978 13254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:16.160123 13254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:16.160177 13254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:16.205091 13254 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:37487
I20260807 19:46:16.205107 13387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:37487 every 8 connection(s)
I20260807 19:46:16.207150 13254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:16.216617 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 13254
I20260807 19:46:16.216823 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:16.217464 795 external_mini_cluster.cc:1428] Setting key a7b82d89ab882e1e53a8181c59a7f9c4
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:46:16.237118 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.210678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:35329 (local address 127.0.198.254:40399)
0807 19:46:16.211319 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.211324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.212688 (+ 1364us) server_negotiation.cc:407] Connection header received
0807 19:46:16.214996 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.215004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.215121 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.215277 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:16.217469 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.218946 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.222898 (+ 3952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.223419 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.223543 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.228791 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:16.228798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:16.228802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:16.228848 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:16.232975 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.234239 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.234252 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.234255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.234398 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.234957 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.234968 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.234970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.235323 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:16.235560 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.236403 (+ 843us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.236688 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":365,"threads_started":1}
I20260807 19:46:16.238267 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.211442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40399 (local address 127.0.198.195:35329)
0807 19:46:16.212469 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.212521 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.214596 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.215613 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.215622 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.216179 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:16.217189 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.217220 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.219233 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.219243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.222189 (+ 2946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.222217 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.223731 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.223742 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.224184 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.225075 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:16.225121 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:16.228440 (+ 3319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:16.233191 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:16.233204 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:16.233234 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:16.234082 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:16.234534 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:16.234540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:16.234544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:16.234812 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:16.235529 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:16.235539 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:16.235966 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.236292 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.236917 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":467,"threads_started":1}
I20260807 19:46:16.240031 13388 heartbeater.cc:347] Connected to a master server at 127.0.198.254:40399
I20260807 19:46:16.240556 13388 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:16.241667 13388 heartbeater.cc:511] Master 127.0.198.254:40399 requested a full tablet report, sending...
I20260807 19:46:16.243770 12876 ts_manager.cc:194] Registered new tserver with Master: 0ec34c295e224d03b257615cc522ddbd (127.0.198.195:37487)
I20260807 19:46:16.244760 12876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:35329
I20260807 19:46:16.247129 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:16.271888 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.252688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.0.198.254:40399)
0807 19:46:16.252948 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.252958 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.253474 (+ 516us) server_negotiation.cc:407] Connection header received
0807 19:46:16.253635 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.253639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.253727 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.253830 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:16.255880 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.256993 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.259058 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.259952 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.260145 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.263092 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:16.263103 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:16.263111 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:16.263169 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:16.267771 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.268960 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.268969 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.268974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.269186 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.269805 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.269813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.269819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.270305 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:16.270608 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.271038 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.271420 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:46:16.278167 12876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:16.293967 12876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:16.362646 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.311533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:37213 (local address 127.0.0.1:48132)
0807 19:46:16.323829 (+ 12296us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.323902 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.324184 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.350654 (+ 26470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.350668 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.350733 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:16.351336 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.351389 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.354797 (+ 3408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.354806 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.358246 (+ 3440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.358269 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.361135 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.361144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.361422 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.361503 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.361643 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.361940 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11900,"thread_start_us":12940,"threads_started":1}
I20260807 19:46:16.363629 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.311889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:34005 (local address 127.0.0.1:59260)
0807 19:46:16.347480 (+ 35591us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.347512 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.347725 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.353390 (+ 5665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.353394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.353421 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:16.353954 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.353970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.358331 (+ 4361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.358337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.360713 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.360729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.362731 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.362737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.362908 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.362952 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.363022 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.363165 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35365,"mutex_wait_us":3,"thread_start_us":40683,"threads_started":1}
I20260807 19:46:16.364339 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.311807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.0.198.193:37213)
0807 19:46:16.349636 (+ 37829us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.349648 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.349688 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:46:16.349853 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.349866 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.350251 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.350540 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:16.351725 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.354563 (+ 2838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.358514 (+ 3951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.360488 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.360662 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.363493 (+ 2831us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.363620 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.363874 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37599,"thread_start_us":39003,"threads_started":1}
I20260807 19:46:16.365062 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.312575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.0.198.194:34005)
0807 19:46:16.351554 (+ 38979us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.351567 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.351611 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:46:16.351746 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.351759 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.352126 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.356379 (+ 4253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:16.356657 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.358149 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.360915 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.362455 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.364178 (+ 1723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.364415 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.364508 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.364731 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38751,"thread_start_us":38609,"threads_started":1}
I20260807 19:46:16.365626 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.315149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:37487 (local address 127.0.0.1:52926)
0807 19:46:16.352889 (+ 37740us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.352910 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.353162 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.354148 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.354152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.354177 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:16.354652 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.354665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.356243 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.356252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.360354 (+ 4102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.360372 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.364675 (+ 4303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.364685 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.364894 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.364927 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.365026 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.365201 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37595,"mutex_wait_us":28,"thread_start_us":33556,"threads_started":1}
I20260807 19:46:16.366776 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.316302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.0.198.195:37487)
0807 19:46:16.348505 (+ 32203us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.348516 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.353027 (+ 4511us) server_negotiation.cc:407] Connection header received
0807 19:46:16.353398 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.353407 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.353666 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.353916 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:16.354867 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.356077 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.361120 (+ 5043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.364008 (+ 2888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.364131 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.364409 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.365912 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.366099 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31914,"thread_start_us":32202,"threads_started":1}
I20260807 19:46:16.369413 13181 tablet_service.cc:1467] Processing CreateTablet for tablet 43949d2b95cf4765bb73ffa64374228d (DEFAULT_TABLE table=test-table [id=c33f59d0eede4e2e9cbb79082ddab609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:16.369551 13030 tablet_service.cc:1467] Processing CreateTablet for tablet 43949d2b95cf4765bb73ffa64374228d (DEFAULT_TABLE table=test-table [id=c33f59d0eede4e2e9cbb79082ddab609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:16.370991 13181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43949d2b95cf4765bb73ffa64374228d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:16.373350 13030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43949d2b95cf4765bb73ffa64374228d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:16.374516 13321 tablet_service.cc:1467] Processing CreateTablet for tablet 43949d2b95cf4765bb73ffa64374228d (DEFAULT_TABLE table=test-table [id=c33f59d0eede4e2e9cbb79082ddab609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:16.376657 13321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43949d2b95cf4765bb73ffa64374228d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:16.397172 13408 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Bootstrap starting.
I20260807 19:46:16.400158 13410 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Bootstrap starting.
I20260807 19:46:16.401331 13409 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Bootstrap starting.
I20260807 19:46:16.405112 13410 tablet_bootstrap.cc:654] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:16.405254 13408 tablet_bootstrap.cc:654] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:16.405589 13409 tablet_bootstrap.cc:654] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:16.406982 13408 log.cc:824] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:16.407670 13409 log.cc:824] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:16.408115 13410 log.cc:824] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:16.417433 13408 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: No bootstrap required, opened a new log
I20260807 19:46:16.417910 13408 ts_tablet_manager.cc:1397] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.000s
I20260807 19:46:16.418659 13410 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: No bootstrap required, opened a new log
I20260807 19:46:16.418912 13409 tablet_bootstrap.cc:492] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: No bootstrap required, opened a new log
I20260807 19:46:16.419021 13410 ts_tablet_manager.cc:1397] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.013s
I20260807 19:46:16.419317 13409 ts_tablet_manager.cc:1397] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.015s
I20260807 19:46:16.430928 13410 raft_consensus.cc:348] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.431730 13410 raft_consensus.cc:374] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:16.431911 13410 raft_consensus.cc:729] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f6779e571c4bbb98b213115f3b7f51, State: Initialized, Role: FOLLOWER
I20260807 19:46:16.432536 13408 raft_consensus.cc:348] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.432850 13410 consensus_queue.cc:260] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.433166 13408 raft_consensus.cc:374] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:16.433320 13408 raft_consensus.cc:729] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a33c0c54814a848efbc179f43da1fa, State: Initialized, Role: FOLLOWER
I20260807 19:46:16.433548 13409 raft_consensus.cc:348] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.434208 13409 raft_consensus.cc:374] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:16.434386 13409 raft_consensus.cc:729] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec34c295e224d03b257615cc522ddbd, State: Initialized, Role: FOLLOWER
I20260807 19:46:16.434306 13408 consensus_queue.cc:260] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.435227 13409 consensus_queue.cc:260] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.440748 13408 ts_tablet_manager.cc:1428] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Time spent starting tablet: real 0.023s user 0.016s sys 0.004s
I20260807 19:46:16.441387 13248 heartbeater.cc:503] Master 127.0.198.254:40399 was elected leader, sending a full tablet report...
I20260807 19:46:16.446358 13409 ts_tablet_manager.cc:1428] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Time spent starting tablet: real 0.025s user 0.020s sys 0.000s
I20260807 19:46:16.446990 13388 heartbeater.cc:503] Master 127.0.198.254:40399 was elected leader, sending a full tablet report...
I20260807 19:46:16.451922 13417 raft_consensus.cc:482] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:16.452370 13417 raft_consensus.cc:504] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.456277 13417 leader_election.cc:288] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f6779e571c4bbb98b213115f3b7f51 (127.0.198.193:37213), 0ec34c295e224d03b257615cc522ddbd (127.0.198.195:37487)
I20260807 19:46:16.459714 13410 ts_tablet_manager.cc:1428] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Time spent starting tablet: real 0.041s user 0.021s sys 0.014s
W20260807 19:46:16.460978 13389 tablet.cc:2369] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:16.465001 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.456795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:48013 (local address 127.0.198.195:37487)
0807 19:46:16.457042 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.457050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.457806 (+ 756us) server_negotiation.cc:407] Connection header received
0807 19:46:16.458076 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.458082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.458160 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.458224 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:16.459069 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.460144 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.462967 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.463928 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.464116 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.464205 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.464579 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.464734 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:46:16.465281 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.456546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:37487 (local address 127.0.198.194:48013)
0807 19:46:16.457697 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.457728 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.457908 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.458446 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.458449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.458477 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:16.458887 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.458902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.460430 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.460434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.462800 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.462813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.464175 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.464181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.464402 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.464428 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.464555 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.464712 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"mutex_wait_us":33,"thread_start_us":5400,"threads_started":1}
I20260807 19:46:16.466318 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43949d2b95cf4765bb73ffa64374228d" candidate_uuid: "c8a33c0c54814a848efbc179f43da1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec34c295e224d03b257615cc522ddbd" is_pre_election: true
I20260807 19:46:16.466856 13342 raft_consensus.cc:2454] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8a33c0c54814a848efbc179f43da1fa in term 0.
I20260807 19:46:16.467623 13133 leader_election.cc:302] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec34c295e224d03b257615cc522ddbd, c8a33c0c54814a848efbc179f43da1fa; no voters:
I20260807 19:46:16.468173 13417 raft_consensus.cc:2781] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:16.468345 13417 raft_consensus.cc:482] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:16.468523 13417 raft_consensus.cc:3037] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:16.468559 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.455470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:55051 (local address 127.0.198.193:37213)
0807 19:46:16.455743 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.455754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.456242 (+ 488us) server_negotiation.cc:407] Connection header received
0807 19:46:16.457374 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.457384 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.457481 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.457572 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:16.458679 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.460103 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.465305 (+ 5202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.466944 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.467114 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.467220 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.468050 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.468242 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":82}
I20260807 19:46:16.468956 13418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.454925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:37213 (local address 127.0.198.194:55051)
0807 19:46:16.456121 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:16.456142 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:16.457213 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:16.457930 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:16.457938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:16.457967 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:16.458375 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.458392 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.462278 (+ 3886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.462286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.465060 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:16.465080 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.467490 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.467499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:16.467806 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.467861 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 19:46:16.467966 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:46:16.468164 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":22,"thread_start_us":879,"threads_started":1}
I20260807 19:46:16.470077 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43949d2b95cf4765bb73ffa64374228d" candidate_uuid: "c8a33c0c54814a848efbc179f43da1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f6779e571c4bbb98b213115f3b7f51" is_pre_election: true
I20260807 19:46:16.470706 13050 raft_consensus.cc:2454] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8a33c0c54814a848efbc179f43da1fa in term 0.
I20260807 19:46:16.472066 13417 raft_consensus.cc:504] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.472838 13417 leader_election.cc:288] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [CANDIDATE]: Term 1 election: Requested vote from peers 90f6779e571c4bbb98b213115f3b7f51 (127.0.198.193:37213), 0ec34c295e224d03b257615cc522ddbd (127.0.198.195:37487)
I20260807 19:46:16.473292 13050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43949d2b95cf4765bb73ffa64374228d" candidate_uuid: "c8a33c0c54814a848efbc179f43da1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f6779e571c4bbb98b213115f3b7f51"
I20260807 19:46:16.473531 13050 raft_consensus.cc:3037] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:16.473727 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43949d2b95cf4765bb73ffa64374228d" candidate_uuid: "c8a33c0c54814a848efbc179f43da1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec34c295e224d03b257615cc522ddbd"
I20260807 19:46:16.473977 13342 raft_consensus.cc:3037] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:16.477048 13050 raft_consensus.cc:2454] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8a33c0c54814a848efbc179f43da1fa in term 1.
I20260807 19:46:16.477327 13342 raft_consensus.cc:2454] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8a33c0c54814a848efbc179f43da1fa in term 1.
I20260807 19:46:16.477872 13133 leader_election.cc:302] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec34c295e224d03b257615cc522ddbd, c8a33c0c54814a848efbc179f43da1fa; no voters:
I20260807 19:46:16.478238 13417 raft_consensus.cc:2781] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:16.478461 13417 raft_consensus.cc:686] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 1 LEADER]: Becoming Leader. State: Replica: c8a33c0c54814a848efbc179f43da1fa, State: Running, Role: LEADER
I20260807 19:46:16.479064 13417 consensus_queue.cc:237] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } }
I20260807 19:46:16.485358 12876 catalog_manager.cc:5495] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa reported cstate change: term changed from 0 to 1, leader changed from <none> to c8a33c0c54814a848efbc179f43da1fa (127.0.198.194). New cstate: current_term: 1 leader_uuid: "c8a33c0c54814a848efbc179f43da1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8a33c0c54814a848efbc179f43da1fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 } health_report { overall_health: UNKNOWN } } }
I20260807 19:46:16.530378 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.522903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.0.198.194:34005)
0807 19:46:16.523139 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.523144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.523234 (+ 90us) server_negotiation.cc:407] Connection header received
0807 19:46:16.523542 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.523545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.523632 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.523769 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:16.524333 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.525151 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.527941 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.528363 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.528463 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.529197 (+ 734us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:16.529743 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:16.529854 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.530015 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.530092 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:46:16.542243 13342 raft_consensus.cc:1261] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 1 FOLLOWER]: Refusing update from remote peer c8a33c0c54814a848efbc179f43da1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:16.542497 13050 raft_consensus.cc:1261] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 1 FOLLOWER]: Refusing update from remote peer c8a33c0c54814a848efbc179f43da1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:16.543596 13415 consensus_queue.cc:1035] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec34c295e224d03b257615cc522ddbd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 37487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:16.544119 13417 consensus_queue.cc:1035] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f6779e571c4bbb98b213115f3b7f51" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:46:16.567157 13249 tablet.cc:2369] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:16.570847 13422 mvcc.cc:205] Tried to move back new op lower bound from 7315996575904235520 to 7315996575663845376. Current Snapshot: MvccSnapshot[applied={T|T < 7315996575904235520}]
I20260807 19:46:16.582000 13423 mvcc.cc:205] Tried to move back new op lower bound from 7315996575904235520 to 7315996575663845376. Current Snapshot: MvccSnapshot[applied={T|T < 7315996575904235520 or (T in {7315996575904235520})}]
I20260807 19:46:16.599779 12876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:16.600106 12876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:16.604727 12876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56889a1cb12146c9898c5002a03e7dd7: Sending DeleteTablet for 3 replicas of tablet 43949d2b95cf4765bb73ffa64374228d
I20260807 19:46:16.606149 13030 tablet_service.cc:1514] Processing DeleteTablet for tablet 43949d2b95cf4765bb73ffa64374228d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:16 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:48132
I20260807 19:46:16.606155 13181 tablet_service.cc:1514] Processing DeleteTablet for tablet 43949d2b95cf4765bb73ffa64374228d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:16 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:59260
I20260807 19:46:16.606660 13321 tablet_service.cc:1514] Processing DeleteTablet for tablet 43949d2b95cf4765bb73ffa64374228d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:16 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:52926
I20260807 19:46:16.610190 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 12958
I20260807 19:46:16.621703 13431 tablet_replica.cc:330] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: stopping tablet replica
I20260807 19:46:16.623216 13431 raft_consensus.cc:2229] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:16.624670 13431 raft_consensus.cc:2258] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:16.627224 13433 tablet_replica.cc:330] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: stopping tablet replica
I20260807 19:46:16.627925 13433 raft_consensus.cc:2229] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:16.628633 13433 raft_consensus.cc:2258] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:16.630108 13431 ts_tablet_manager.cc:1905] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:16.635241 13431 ts_tablet_manager.cc:1918] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:16.636049 13431 log.cc:1196] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/43949d2b95cf4765bb73ffa64374228d
I20260807 19:46:16.636380 13432 tablet_replica.cc:330] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: stopping tablet replica
I20260807 19:46:16.636698 13431 ts_tablet_manager.cc:1939] T 43949d2b95cf4765bb73ffa64374228d P 90f6779e571c4bbb98b213115f3b7f51: Deleting consensus metadata
I20260807 19:46:16.636874 13432 raft_consensus.cc:2229] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:46:16.637562 13432 raft_consensus.cc:2258] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:16.637879 13433 ts_tablet_manager.cc:1905] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:16.638936 12862 catalog_manager.cc:4841] TS 90f6779e571c4bbb98b213115f3b7f51 (127.0.198.193:37213): tablet 43949d2b95cf4765bb73ffa64374228d (table test-table [id=c33f59d0eede4e2e9cbb79082ddab609]) successfully deleted
I20260807 19:46:16.641228 13432 ts_tablet_manager.cc:1905] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:46:16.646430 13433 ts_tablet_manager.cc:1918] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:16.646739 13433 log.cc:1196] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/43949d2b95cf4765bb73ffa64374228d
I20260807 19:46:16.647392 13433 ts_tablet_manager.cc:1939] T 43949d2b95cf4765bb73ffa64374228d P 0ec34c295e224d03b257615cc522ddbd: Deleting consensus metadata
I20260807 19:46:16.649415 13432 ts_tablet_manager.cc:1918] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:16.649713 13432 log.cc:1196] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/43949d2b95cf4765bb73ffa64374228d
I20260807 19:46:16.649649 12860 catalog_manager.cc:4841] TS 0ec34c295e224d03b257615cc522ddbd (127.0.198.195:37487): tablet 43949d2b95cf4765bb73ffa64374228d (table test-table [id=c33f59d0eede4e2e9cbb79082ddab609]) successfully deleted
I20260807 19:46:16.650354 13432 ts_tablet_manager.cc:1939] T 43949d2b95cf4765bb73ffa64374228d P c8a33c0c54814a848efbc179f43da1fa: Deleting consensus metadata
I20260807 19:46:16.652519 12863 catalog_manager.cc:4841] TS c8a33c0c54814a848efbc179f43da1fa (127.0.198.194:34005): tablet 43949d2b95cf4765bb73ffa64374228d (table test-table [id=c33f59d0eede4e2e9cbb79082ddab609]) successfully deleted
I20260807 19:46:16.655517 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.613013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48140 (local address 127.0.198.193:37213)
0807 19:46:16.613273 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.613283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.626350 (+ 13067us) server_negotiation.cc:407] Connection header received
0807 19:46:16.626599 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.626609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.626745 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.626877 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:16.628637 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.629737 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.640529 (+ 10792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.641223 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.641352 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.647892 (+ 6540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:16.647916 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:16.647937 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:16.647993 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:16.651911 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.653039 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.653049 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.653051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.653197 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.653660 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.653664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.653666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.654091 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:16.654435 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.654811 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.655085 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:46:16.656257 13095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:16.750864 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 12958
I20260807 19:46:16.771876 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 13106
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:46:16.791761 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.772957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.0.198.194:34005)
0807 19:46:16.773178 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.773185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.773207 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:46:16.773276 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.773284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.773386 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.773502 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:16.775076 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.776273 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.777837 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.778648 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.778788 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.783949 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:16.783969 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:16.783995 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:16.784105 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:16.788149 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.789061 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.789074 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.789077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.789318 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.789812 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.789822 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.789824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.790251 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:16.790653 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.790990 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.791315 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:46:16.792629 13243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:16.914367 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 13106
I20260807 19:46:16.937788 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 13254
Aug 07 19:46:16 dist-test-slave-btw8 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:46:16.956060 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:16.938734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.0.198.195:37487)
0807 19:46:16.938931 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:16.938937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:16.939082 (+ 145us) server_negotiation.cc:407] Connection header received
0807 19:46:16.939197 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:16.939202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:16.939279 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:16.939762 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:16.941004 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.941825 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.943806 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:16.944254 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:16.944381 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:16.949619 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:16.949687 (+ 68us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:16.949705 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:16.949751 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:16.952893 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.953759 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.953767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.953770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.953900 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:16.954339 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:16.954348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:16.954351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:16.954887 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:16.955150 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:16.955507 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 19:46:16.955759 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:46:16.956826 13383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:17.053948 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 13254
I20260807 19:46:17.075086 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 12843
I20260807 19:46:17.076768 12906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:17.201298 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 12843
I20260807 19:46:17.230342 795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:46:17.578018 795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:46:17.578336 795 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:46:17.941376 795 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:46:17.941 UTC [12410] LOG: received smart shutdown request
2026-08-07 19:46:17.946 UTC [12410] LOG: background worker "logical replication launcher" (PID 12419) exited with exit code 1
2026-08-07 19:46:17.946 UTC [12413] LOG: shutting down
2026-08-07 19:46:18.159 UTC [12410] LOG: database system is shut down
2026-08-07T19:46:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57080 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:46:18 dist-test-slave-btw8 krb5kdc[13447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:46:18 dist-test-slave-btw8 krb5kdc[13447](info): set up 2 sockets
Aug 07 19:46:18 dist-test-slave-btw8 krb5kdc[13447](info): commencing operation
krb5kdc: starting...
W20260807 19:46:19.616664 795 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.007s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:46:19 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:46:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangeradmin_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangeradmin_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangerlookup_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangerlookup_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
I20260807 19:46:20.083904 795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYvfPwA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/postgres -l logfile start
2026-08-07 19:46:23.122 UTC [13477] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:46:23.123 UTC [13477] LOG: listening on IPv4 address "127.0.198.212", port 49429
2026-08-07 19:46:23.127 UTC [13477] LOG: listening on Unix socket "/tmp/.s.PGSQL.49429"
2026-08-07 19:46:23.154 UTC [13479] LOG: database system was shut down at 2026-08-07 19:46:21 UTC
2026-08-07 19:46:23.158 UTC [13477] LOG: database system is ready to accept connections
I20260807 19:46:24.225809 795 mini_postgres.cc:96] Postgres bound to 49429
2026-08-07 19:46:24.253 UTC [13489] FATAL: database "slave" does not exist
127.0.198.212:49429 - accepting connections
I20260807 19:46:24.254146 795 mini_ranger.cc:161] Starting Ranger...
I20260807 19:46:24.290782 795 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:46:24.921689 795 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:46:24.921842 795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin
2026-08-07 19:46:25,234 [I] DB FLAVOR :POSTGRES
2026-08-07 19:46:25,234 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:46:25,234 [I] Checking connection..
2026-08-07 19:46:25,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:46:25,502 [I] Checking connection passed.
2026-08-07 19:46:25,503 [I] --------- Verifying version history table ---------
2026-08-07 19:46:25,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:46:25,805 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:46:25,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:46:26,095 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:46:26,095 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:46:26,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:46:26.356 UTC [13555] WARNING: there is no transaction in progress
2026-08-07 19:46:26,365 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:46:26,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:46:26,666 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:46:26,667 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:46:26,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:46:26,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:46:27,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-btw8.c.gcp-upstream.internal','N') ;"
2026-08-07 19:46:27,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:46:27,826 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:46:27,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:46:28,131 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:46:28,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:46:28,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:46:28,708 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:46:28,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:46:29.850 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:29.869 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:29.890 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:29.911 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:29.931 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:29.952 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.016 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.026 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.044 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.058 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.070 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.089 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.102 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.111 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.120 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30.130 UTC [13670] WARNING: there is no transaction in progress
2026-08-07 19:46:30,988 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:46:30,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:46:31,256 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:46:31,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:46:31,544 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:46:31,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:46:31,833 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:46:31,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:46:32,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:46:32,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-btw8.c.gcp-upstream.internal';"
2026-08-07 19:46:32,660 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:46:32,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:46:32,954 [I] DB_PATCHES have already been applied
I20260807 19:46:32.960819 795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYvfPwA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:46:32.960916 795 mini_ranger.cc:193] Using host: 127.0.198.212
I20260807 19:46:32.984654 795 mini_ranger.cc:227] Ranger admin URL: http://127.0.198.212:58397
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:46:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:46:34.232029 795 mini_ranger.cc:160] Time spent starting Ranger: real 9.978s user 0.000s sys 0.075s
Aug 07, 2026 7:46:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:46:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 7:46:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:46:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:46:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:46:59.239478 795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.198.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.198.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangerkms_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.198.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/rangerkms_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.198.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.198.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.198.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/HTTP_127.0.198.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:46:59 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:46:59.503808 795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:46:59.544106 795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:47:00.675416 795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:47:00.675551 795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms
I20260807 19:47:00.675696 795 mini_ranger_kms.cc:227] Using postgres at 127.0.198.212:49429
2026-08-07 19:47:01,068 [I] DB FLAVOR :POSTGRES
2026-08-07 19:47:01,069 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:47:01,069 [I] Checking connection
2026-08-07 19:47:01,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:47:01,338 [I] connection success
2026-08-07 19:47:01,338 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:47:01,338 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:47:01,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:47:01,613 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:47:01,613 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:47:01,613 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:47:01,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.198.212:49429/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:47:01,922 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:47:02.453022 795 mini_ranger_kms.cc:310] Created kms service
I20260807 19:47:02.667673 795 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:47:02.720628 795 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:47:02 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:03.210227 795 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:47:03.210572 795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYvfPwA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYvfPwA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:47:03.210611 795 mini_ranger_kms.cc:240] Using host: 127.0.198.212
I20260807 19:47:03.239907 795 mini_ranger_kms.cc:276] Ranger KMS PID: 13861
I20260807 19:47:03.240111 795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.198.212:56079
Aug 07, 2026 7:47:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:47:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:47:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:47:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 19:47:05.736881 795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.233s user 0.096s sys 0.340s
I20260807 19:47:05.737160 795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:47:05.737344 795 mini_ranger_kms.cc:401] 127.0.198.212:56079/kms/v1/keys
Aug 07 19:47:05 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786132025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 19:47:06 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
Aug 07 19:47:06 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.198.212@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
Aug 07 19:47:09 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:47:09.403705 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:46193
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:46193
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:09.770668 13914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:09.771004 13914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:09.771096 13914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:09.778903 13914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:09.779049 13914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:09.779098 13914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:09.779150 13914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:09.779199 13914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:09.790194 13914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.198.212:56079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:46193
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:09.791628 13914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:09.793077 13914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:09.806865 13919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:09.806900 13920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:09.806833 13922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:09.807391 13914 server_base.cc:1034] running on GCE node
I20260807 19:47:09.808548 13914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:09.810354 13914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:09.811713 13914 hybrid_clock.cc:648] HybridClock initialized: now 1786132029811670 us; error 62 us; skew 500 ppm
I20260807 19:47:09.812183 13914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:09 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:09.816043 13914 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:09.817821 13914 webserver.cc:460] Webserver started at http://127.0.198.254:36321/ using document root <none> and password file <none>
I20260807 19:47:09.818532 13914 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:09.818677 13914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:09.819029 13914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:47:09 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:09.890972 13914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6"
format_stamp: "Formatted at 2026-08-07 19:47:09 on dist-test-slave-btw8"
server_key: "cbb9e8f2bd736b48334da37e848b163a"
server_key_iv: "e2cfbc4b80277a4edfb52ced0f5b2afe"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:09.892027 13914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6"
format_stamp: "Formatted at 2026-08-07 19:47:09 on dist-test-slave-btw8"
server_key: "cbb9e8f2bd736b48334da37e848b163a"
server_key_iv: "e2cfbc4b80277a4edfb52ced0f5b2afe"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:09.898586 13914 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:47:09.922564 13929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:09.924710 13914 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:47:09.924947 13914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6"
format_stamp: "Formatted at 2026-08-07 19:47:09 on dist-test-slave-btw8"
server_key: "cbb9e8f2bd736b48334da37e848b163a"
server_key_iv: "e2cfbc4b80277a4edfb52ced0f5b2afe"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:09.925240 13914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:09.949513 13914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:09.953718 13914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:09.954001 13914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:09.974859 13914 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:46193
I20260807 19:47:09.974892 13981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:46193 every 8 connection(s)
I20260807 19:47:09.976403 13914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:09.980635 13982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:09.982523 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 13914
I20260807 19:47:09.982806 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:47:09.983740 795 external_mini_cluster.cc:1428] Setting key e193c2d89759416219678954aea13c10
I20260807 19:47:09.994272 13982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Bootstrap starting.
I20260807 19:47:09.998112 13982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:09.999228 13982 log.cc:824] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:47:09 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:10.003736 13982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: No bootstrap required, opened a new log
I20260807 19:47:10.012009 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:09.986132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.0.198.254:46193)
0807 19:47:09.987181 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:09.987200 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:09.987254 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:47:09.989314 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:09.989341 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:09.989797 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:09.990251 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:09.992035 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:09.993513 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:09.995116 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:09.995987 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:09.996246 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:10.001694 (+ 5448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:10.001709 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:10.001731 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:10.001778 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:10.006430 (+ 4652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.007591 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.007606 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.007616 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.007827 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.008296 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.008302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.008305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.009106 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:10.009453 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:10.009935 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 19:47:10.010431 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":491,"threads_started":1}
I20260807 19:47:10.015048 13982 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:10.015627 13982 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:10.015789 13982 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf587eaa6784ce88c3b247dd9cdc3a6, State: Initialized, Role: FOLLOWER
I20260807 19:47:10.016427 13982 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:10.016686 13982 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:10.016857 13982 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:10.017040 13982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:10.019567 13982 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:10.020155 13982 leader_election.cc:302] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddf587eaa6784ce88c3b247dd9cdc3a6; no voters:
I20260807 19:47:10.020874 13982 leader_election.cc:288] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:10.021044 13987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:10.022387 13987 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 LEADER]: Becoming Leader. State: Replica: ddf587eaa6784ce88c3b247dd9cdc3a6, State: Running, Role: LEADER
I20260807 19:47:10.023135 13987 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:10.023542 13982 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:10.028007 13989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddf587eaa6784ce88c3b247dd9cdc3a6. Latest consensus state: current_term: 1 leader_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } } }
I20260807 19:47:10.027830 13988 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } } }
I20260807 19:47:10.028491 13988 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:10.028491 13989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:10.032979 13997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:10.038887 13997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:10.039727 13982 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:47:10.061828 13997 catalog_manager.cc:1347] Generated new cluster ID: 644c7ec49cb647ad85068281d651bad2
I20260807 19:47:10.062155 13997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:10.101877 13997 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:10.104245 13997 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:10.114578 13997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Generated new TSK 0
I20260807 19:47:10.116008 13997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:47:10.908819 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:11 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:47:11.406531 14029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:11.406988 14029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:11.407226 14029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:11.420042 14029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:11.420305 14029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:11.420545 14029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:11.437794 14029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:11.439941 14029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:11.441835 14029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:11.462908 14040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:11.463105 14042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:11.463884 14039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:11.464717 14029 server_base.cc:1034] running on GCE node
I20260807 19:47:11.465483 14029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:11.466607 14029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:11.467860 14029 hybrid_clock.cc:648] HybridClock initialized: now 1786132031467790 us; error 67 us; skew 500 ppm
I20260807 19:47:11.468458 14029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:11 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:11.472378 14029 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:11.474320 14029 webserver.cc:460] Webserver started at http://127.0.198.193:42463/ using document root <none> and password file <none>
I20260807 19:47:11.475255 14029 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:11.475464 14029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:11.475838 14029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:47:11 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:11.514635 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "0c502e14bfb3425590faab588343a35a"
format_stamp: "Formatted at 2026-08-07 19:47:11 on dist-test-slave-btw8"
server_key: "49fc557790ab0214eb06d95094b82b09"
server_key_iv: "791d21deefec5958b03aeba018cd77af"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:11.516741 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "0c502e14bfb3425590faab588343a35a"
format_stamp: "Formatted at 2026-08-07 19:47:11 on dist-test-slave-btw8"
server_key: "49fc557790ab0214eb06d95094b82b09"
server_key_iv: "791d21deefec5958b03aeba018cd77af"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:11.528218 14029 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260807 19:47:11.569963 14049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:11.572568 14029 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.014s sys 0.004s
I20260807 19:47:11.572958 14029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "0c502e14bfb3425590faab588343a35a"
format_stamp: "Formatted at 2026-08-07 19:47:11 on dist-test-slave-btw8"
server_key: "49fc557790ab0214eb06d95094b82b09"
server_key_iv: "791d21deefec5958b03aeba018cd77af"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:11.573396 14029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:11.669780 14029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:11.675693 14029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:11.676139 14029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:11.693859 14029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:11.700407 14029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:11.700629 14029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:47:11.700865 14029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:11.701009 14029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:47:11 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:11.871037 14029 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:43631
I20260807 19:47:11.871045 14166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:43631 every 8 connection(s)
I20260807 19:47:11.872705 14029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:11.883010 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14029
I20260807 19:47:11.883296 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:47:11.884318 795 external_mini_cluster.cc:1428] Setting key 63d67f5dba81283ec12cf37abe920123
Aug 07 19:47:11 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:11.937798 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:11.878273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:35105 (local address 127.0.198.254:46193)
0807 19:47:11.908021 (+ 29748us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:11.908030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:11.911431 (+ 3401us) server_negotiation.cc:407] Connection header received
0807 19:47:11.915159 (+ 3728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:11.915166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:11.915297 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:11.915456 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:11.917954 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:11.919212 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:11.922064 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:11.922777 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:11.922903 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:11.928336 (+ 5433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:11.928345 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:11.928350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:11.928407 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:11.932720 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:11.934113 (+ 1393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:11.934123 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:11.934127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:11.934361 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:11.935067 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:11.935073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:11.935076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:11.935650 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:11.935801 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:11.937055 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0807 19:47:11.937366 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29579,"thread_start_us":29737,"threads_started":1}
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:47:11.940678 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:11.877306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.193:35105)
0807 19:47:11.908901 (+ 31595us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:11.908970 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:11.913449 (+ 4479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:11.916001 (+ 2552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:11.916014 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:11.916729 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:11.917684 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:11.917715 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:11.919533 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:11.919538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:11.921814 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:11.921850 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:11.923414 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:11.923425 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:11.923861 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:11.924791 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:11.924830 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:11.927882 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:11.933035 (+ 5153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:11.933049 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:11.933073 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:11.933899 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:11.934571 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:11.934577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:11.934580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:11.934893 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:11.935946 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:11.935955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:11.936520 (+ 565us) client_negotiation.cc:769] Sending connection context
0807 19:47:11.938349 (+ 1829us) client_negotiation.cc:240] Negotiation successful
0807 19:47:11.939017 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31145,"thread_start_us":33928,"threads_started":1}
I20260807 19:47:11.943102 14167 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:11.943533 14167 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:47:11.944674 14167 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:11.947615 13946 ts_manager.cc:194] Registered new tserver with Master: 0c502e14bfb3425590faab588343a35a (127.0.198.193:43631)
I20260807 19:47:11.949779 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:35105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
W20260807 19:47:11.992264 795 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.198.194: real 0.104s user 0.000s sys 0.030s
I20260807 19:47:11.993328 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:12 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
W20260807 19:47:12.507019 14174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:12.507507 14174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:12.507717 14174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:12.516172 14174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:12.516338 14174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:12.516531 14174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:12.527206 14174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:12.528728 14174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:12.530144 14174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:12.545069 14189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:12.545109 14192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:12.545142 14190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:12.545886 14174 server_base.cc:1034] running on GCE node
I20260807 19:47:12.546568 14174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:12.547806 14174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:12.549079 14174 hybrid_clock.cc:648] HybridClock initialized: now 1786132032549031 us; error 42 us; skew 500 ppm
I20260807 19:47:12.549433 14174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:12 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:12.553035 14174 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:47:12.554874 14174 webserver.cc:460] Webserver started at http://127.0.198.194:37189/ using document root <none> and password file <none>
I20260807 19:47:12.555733 14174 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:12.555853 14174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:12.556139 14174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:47:12 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:12.573529 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "9d95ca7199254483948372e200c4f98d"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-btw8"
server_key: "de88a24adb5d10ae0f0cf55fccb5b0ff"
server_key_iv: "b46552ed9ebffb95178f60e6d935fa0d"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:12.574482 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "9d95ca7199254483948372e200c4f98d"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-btw8"
server_key: "de88a24adb5d10ae0f0cf55fccb5b0ff"
server_key_iv: "b46552ed9ebffb95178f60e6d935fa0d"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:12.580816 14174 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260807 19:47:12.599160 14199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:12.601092 14174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:47:12.601307 14174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "9d95ca7199254483948372e200c4f98d"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-btw8"
server_key: "de88a24adb5d10ae0f0cf55fccb5b0ff"
server_key_iv: "b46552ed9ebffb95178f60e6d935fa0d"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:12.601596 14174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:12.638296 14174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:12.644174 14174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:12.644560 14174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:12.645870 14174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:12.647848 14174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:12.647948 14174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:12.648080 14174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:12.648165 14174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:12.687492 14174 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35785
I20260807 19:47:12.687536 14314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35785 every 8 connection(s)
I20260807 19:47:12.689288 14174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:12.698843 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14174
I20260807 19:47:12.699101 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:47:12.699795 795 external_mini_cluster.cc:1428] Setting key f4a28860f1773a842526df75e69f9ad5
Aug 07 19:47:12 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:12.716512 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:12.692268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:33669 (local address 127.0.198.254:46193)
0807 19:47:12.692794 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:12.692801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:12.693948 (+ 1147us) server_negotiation.cc:407] Connection header received
0807 19:47:12.695900 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:12.695907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:12.695997 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:12.696143 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:12.697906 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.699049 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.701607 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.702266 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.702432 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:12.707955 (+ 5523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:12.707964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:12.707991 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:12.708060 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:12.711957 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:12.713267 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:12.713275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:12.713279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:12.713488 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:12.714159 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:12.714165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:12.714168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:12.714564 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:12.714739 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:12.715850 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0807 19:47:12.716120 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":257,"threads_started":1}
I20260807 19:47:12.718137 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:12.692869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.194:33669)
0807 19:47:12.693760 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:12.693805 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:12.695584 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:12.696443 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:12.696453 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:12.697016 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:12.697674 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:12.697709 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.699283 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.699293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:12.701423 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:12.701440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.702545 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.702553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:12.702933 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:12.703908 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:12.703942 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:12.707252 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:12.712185 (+ 4933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:12.712199 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:12.712225 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:12.713067 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:12.713627 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:12.713636 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:12.713644 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:12.713985 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:12.714728 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:12.714741 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:12.715282 (+ 541us) client_negotiation.cc:769] Sending connection context
0807 19:47:12.715697 (+ 415us) client_negotiation.cc:240] Negotiation successful
0807 19:47:12.716394 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":401,"threads_started":1}
I20260807 19:47:12.720009 14315 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:12.720382 14315 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:12.721396 14315 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:12.723446 13946 ts_manager.cc:194] Registered new tserver with Master: 9d95ca7199254483948372e200c4f98d (127.0.198.194:35785)
I20260807 19:47:12.724972 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:33669
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:47:12.780637 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:12.954159 14167 heartbeater.cc:503] Master 127.0.198.254:46193 was elected leader, sending a full tablet report...
W20260807 19:47:13.151397 14321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:13.151811 14321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:13.151989 14321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:13.159993 14321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:13.160184 14321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:13.160375 14321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:13.171221 14321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:13.172757 14321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:13.174124 14321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:13.189241 14327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:13.189361 14326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:13.189386 14330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:13.190160 14321 server_base.cc:1034] running on GCE node
I20260807 19:47:13.190852 14321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:13.192035 14321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:13.193252 14321 hybrid_clock.cc:648] HybridClock initialized: now 1786132033193174 us; error 71 us; skew 500 ppm
I20260807 19:47:13.193615 14321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:13.197861 14321 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:13.199957 14321 webserver.cc:460] Webserver started at http://127.0.198.195:40595/ using document root <none> and password file <none>
I20260807 19:47:13.200989 14321 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:13.201160 14321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:13.201573 14321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:13.219063 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "acbb8c1c10194f15879782368a6df3dd"
format_stamp: "Formatted at 2026-08-07 19:47:13 on dist-test-slave-btw8"
server_key: "f6598c7d13643e784652bd8111f6da40"
server_key_iv: "8dce92efba755913f904309a63377a03"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:13.220096 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "acbb8c1c10194f15879782368a6df3dd"
format_stamp: "Formatted at 2026-08-07 19:47:13 on dist-test-slave-btw8"
server_key: "f6598c7d13643e784652bd8111f6da40"
server_key_iv: "8dce92efba755913f904309a63377a03"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:13.226645 14321 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:47:13.245901 14337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:13.248051 14321 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 19:47:13.248287 14321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "acbb8c1c10194f15879782368a6df3dd"
format_stamp: "Formatted at 2026-08-07 19:47:13 on dist-test-slave-btw8"
server_key: "f6598c7d13643e784652bd8111f6da40"
server_key_iv: "8dce92efba755913f904309a63377a03"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:13.248569 14321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:13.268128 14321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:13.272925 14321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:13.273226 14321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:13.274317 14321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:13.276078 14321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:13.276166 14321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:13.276265 14321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:13.276315 14321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:13.316756 14321 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36925
I20260807 19:47:13.316866 14457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36925 every 8 connection(s)
I20260807 19:47:13.318148 14321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:13.326197 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14321
I20260807 19:47:13.326460 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:47:13.327142 795 external_mini_cluster.cc:1428] Setting key dc73a657394e14526c7897ab3bdcf06a
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:13.343343 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:13.321309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:35159 (local address 127.0.198.254:46193)
0807 19:47:13.321936 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:13.321943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:13.322933 (+ 990us) server_negotiation.cc:407] Connection header received
0807 19:47:13.325266 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:13.325272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:13.325649 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:13.325845 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:13.328024 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.329206 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.331302 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.331878 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.332025 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:13.336089 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:13.336095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:13.336099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:13.336162 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:13.339028 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.340235 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.340242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.340245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.340415 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.341004 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.341018 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.341028 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.341407 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:13.341597 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:13.342583 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 19:47:13.342884 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":281,"threads_started":1}
I20260807 19:47:13.344487 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:13.321875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.195:35159)
0807 19:47:13.322736 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:13.322778 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:13.324837 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:13.326178 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:13.326189 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:13.326880 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:13.327715 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:13.327738 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.329396 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.329401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:13.331125 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:13.331139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.332138 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.332148 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:13.332447 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:13.333238 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:13.333278 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:13.335596 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:13.339277 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:13.339286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:13.339303 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:13.340019 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:13.340579 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:13.340585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:13.340588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:13.340851 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:13.341591 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:13.341601 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:13.342059 (+ 458us) client_negotiation.cc:769] Sending connection context
0807 19:47:13.342466 (+ 407us) client_negotiation.cc:240] Negotiation successful
0807 19:47:13.343085 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":355,"threads_started":1}
I20260807 19:47:13.346503 14458 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:13.346901 14458 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:13.347805 14458 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:13.349582 13946 ts_manager.cc:194] Registered new tserver with Master: acbb8c1c10194f15879782368a6df3dd (127.0.198.195:36925)
I20260807 19:47:13.350476 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:35159
I20260807 19:47:13.357583 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:13.359637 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14029
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:47:13.379207 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:13.360861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.0.198.193:43631)
0807 19:47:13.361602 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:13.361610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:13.361639 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:47:13.361704 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:13.361711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:13.361936 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:13.362076 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:13.363788 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.365016 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.366739 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.367471 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.367598 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:13.372522 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:13.372540 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:13.372557 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:13.372624 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:13.375997 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.376958 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.376966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.376968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.377183 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.377610 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.377616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.377619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.378038 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:13.378339 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:13.378602 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 19:47:13.378861 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":359,"threads_started":1}
I20260807 19:47:13.380851 14162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:13.469219 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14029
I20260807 19:47:13.488881 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14174
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:47:13.513777 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:13.489921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.0.198.194:35785)
0807 19:47:13.490535 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:13.490543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:13.490569 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:47:13.490661 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:13.490669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:13.490894 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:13.491095 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:13.492714 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.493867 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.495558 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.496188 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.496351 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:13.506319 (+ 9968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:13.506338 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:13.506355 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:13.506405 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:13.510140 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.511116 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.511124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.511127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.511307 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.511835 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.511849 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.511852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.512267 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:13.512637 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:13.513017 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:47:13.513340 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":354,"threads_started":1}
I20260807 19:47:13.515317 14310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:13.601267 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14174
I20260807 19:47:13.620105 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14321
Aug 07 19:47:13 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:47:13.641067 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:13.621404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.0.198.195:36925)
0807 19:47:13.622145 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:13.622153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:13.622185 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:47:13.622249 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:13.622256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:13.622497 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:13.622716 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:13.624415 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.625606 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.627481 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:13.628130 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:13.628253 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:13.633863 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:13.633880 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:13.633899 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:13.633952 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:13.637487 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.638391 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.638401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.638403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.638568 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:13.639151 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:13.639156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:13.639158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:13.639771 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:13.640080 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:13.640426 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 19:47:13.640730 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":411,"threads_started":1}
I20260807 19:47:13.642660 14453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:13.736126 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14321
I20260807 19:47:13.756036 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 13914
I20260807 19:47:13.757583 13977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:13.872566 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 13914
I20260807 19:47:13.906376 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:46193
--webserver_interface=127.0.198.254
--webserver_port=36321
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:46193
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:14.268666 14470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:14.269096 14470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:14.269260 14470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:14.277110 14470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:14.277308 14470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:14.277427 14470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:14.277526 14470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:14.277614 14470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:14.288887 14470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:46193
--ranger_config_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:14.290364 14470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:14.291847 14470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:14.304804 14478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:14.304858 14475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:14.304873 14476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:14.305886 14470 server_base.cc:1034] running on GCE node
I20260807 19:47:14.306972 14470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:14.309190 14470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:14.310510 14470 hybrid_clock.cc:648] HybridClock initialized: now 1786132034310430 us; error 71 us; skew 500 ppm
I20260807 19:47:14.311017 14470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:14 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:14.318734 14470 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:14.320506 14470 webserver.cc:460] Webserver started at http://127.0.198.254:36321/ using document root <none> and password file <none>
I20260807 19:47:14.321211 14470 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:14.321333 14470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:47:14 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:14.411562 14470 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:47:14.415756 14487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:14.417675 14470 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:47:14.417905 14470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6"
format_stamp: "Formatted at 2026-08-07 19:47:09 on dist-test-slave-btw8"
server_key: "cbb9e8f2bd736b48334da37e848b163a"
server_key_iv: "e2cfbc4b80277a4edfb52ced0f5b2afe"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:14.419344 14470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:14.435578 14470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:14.439782 14470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:14.440039 14470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:14.461743 14470 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:46193
I20260807 19:47:14.461781 14544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:46193 every 8 connection(s)
I20260807 19:47:14.463411 14470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:14.472151 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14470
I20260807 19:47:14.472172 14545 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:47:14.473096 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:43631
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:46193
--webserver_port=42463
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:14.476421 14545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Bootstrap starting.
I20260807 19:47:14.498771 14545 log.cc:824] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:14.509452 14545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:47:14.510157 14545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Bootstrap complete.
I20260807 19:47:14.519902 14545 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:14.520960 14545 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf587eaa6784ce88c3b247dd9cdc3a6, State: Initialized, Role: FOLLOWER
I20260807 19:47:14.521697 14545 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:14.521965 14545 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:14.522099 14545 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:14.522297 14545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:14.527307 14545 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:14.527956 14545 leader_election.cc:302] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddf587eaa6784ce88c3b247dd9cdc3a6; no voters:
I20260807 19:47:14.528666 14545 leader_election.cc:288] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:47:14.528913 14549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:14.530181 14549 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [term 2 LEADER]: Becoming Leader. State: Replica: ddf587eaa6784ce88c3b247dd9cdc3a6, State: Running, Role: LEADER
I20260807 19:47:14.531095 14545 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:14.530938 14549 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } }
I20260807 19:47:14.537710 14550 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } } }
I20260807 19:47:14.538380 14550 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:14.539302 14551 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddf587eaa6784ce88c3b247dd9cdc3a6. Latest consensus state: current_term: 2 leader_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf587eaa6784ce88c3b247dd9cdc3a6" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 46193 } } }
I20260807 19:47:14.539837 14551 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:14.548048 14559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:14.556499 14559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:14.558905 14559 catalog_manager.cc:1259] Loaded cluster ID: 644c7ec49cb647ad85068281d651bad2
I20260807 19:47:14.559203 14559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:14.564502 14545 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:47:14.565605 14559 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:14.568045 14559 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Loaded TSK: 0
I20260807 19:47:14.570389 14559 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:47:14.992743 14547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:14.993243 14547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:14.993438 14547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:15.007256 14547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:15.007459 14547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:15.007686 14547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:15.026286 14547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:43631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=42463
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:15.028262 14547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:15.030263 14547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:15.065188 14585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:15.065850 14587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:15.067843 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:15.070688 14547 server_base.cc:1034] running on GCE node
I20260807 19:47:15.071746 14547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:15.073376 14547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:15.074682 14547 hybrid_clock.cc:648] HybridClock initialized: now 1786132035074585 us; error 81 us; skew 500 ppm
I20260807 19:47:15.075201 14547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:15.088404 14547 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:15.101388 14547 webserver.cc:460] Webserver started at http://127.0.198.193:42463/ using document root <none> and password file <none>
I20260807 19:47:15.102722 14547 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:15.102933 14547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:15.154086 14547 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.002s
I20260807 19:47:15.165594 14596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:15.167924 14547 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.007s sys 0.004s
I20260807 19:47:15.168248 14547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "0c502e14bfb3425590faab588343a35a"
format_stamp: "Formatted at 2026-08-07 19:47:11 on dist-test-slave-btw8"
server_key: "49fc557790ab0214eb06d95094b82b09"
server_key_iv: "791d21deefec5958b03aeba018cd77af"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:15.169988 14547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:15.222476 14547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:15.228919 14547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:15.229420 14547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:15.230865 14547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:15.236405 14547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:15.236624 14547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:15.236794 14547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:15.236898 14547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:15.297657 14547 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:43631
I20260807 19:47:15.297711 14713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:43631 every 8 connection(s)
I20260807 19:47:15.299185 14547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:15.301213 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14547
I20260807 19:47:15.302089 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:35785
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:46193
--webserver_port=37189
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:15.346099 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:15.303207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.193:56383)
0807 19:47:15.313035 (+ 9828us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:15.313131 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:15.318635 (+ 5504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:15.321627 (+ 2992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:15.321640 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:15.322430 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:15.323425 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:15.323475 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:15.326741 (+ 3266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:15.326760 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:15.329025 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:15.329061 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:15.330669 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:15.330678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:15.330982 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:15.331839 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:15.331877 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:15.334597 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:15.339807 (+ 5210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:15.339818 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:15.339837 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:15.340559 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:15.341228 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:15.341234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:15.341238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:15.341560 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:15.342850 (+ 1290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:15.342862 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:15.343389 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 19:47:15.343770 (+ 381us) client_negotiation.cc:240] Negotiation successful
0807 19:47:15.344490 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9394,"thread_start_us":12060,"threads_started":1}
I20260807 19:47:15.349545 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:15.303184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:56383 (local address 127.0.198.254:46193)
0807 19:47:15.319411 (+ 16227us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:15.319434 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:15.319490 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:47:15.320015 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:15.320039 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:15.320615 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:15.323810 (+ 3195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:15.324289 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:15.326519 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:15.329328 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:15.330388 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:15.330641 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:15.335070 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:15.335087 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:15.335113 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:15.335179 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:15.339576 (+ 4397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:15.340800 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:15.340810 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:15.340818 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:15.341071 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:15.341744 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:15.341749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:15.341753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:15.342647 (+ 894us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:15.347846 (+ 5199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:15.348124 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 19:47:15.348511 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15832,"thread_start_us":17660,"threads_started":1}
I20260807 19:47:15.353320 14714 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:15.353693 14714 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:15.354494 14714 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
W20260807 19:47:15.357599 14714 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:46193 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:47:15.357859 14714 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:15.358155 14714 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:15.359498 14714 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:15.359827 14714 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
W20260807 19:47:15.360915 14714 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:47:15.841439 14719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:15.841826 14719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:15.841992 14719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:15.851032 14719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:15.851222 14719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:15.851487 14719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:15.862496 14719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:35785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=37189
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:15.864210 14719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:15.865669 14719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:15.885365 14736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:15.886109 14735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:15.891160 14719 server_base.cc:1034] running on GCE node
W20260807 19:47:15.891691 14738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:15.892516 14719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:15.893878 14719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:15.897322 14719 hybrid_clock.cc:648] HybridClock initialized: now 1786132035897233 us; error 62 us; skew 500 ppm
I20260807 19:47:15.897794 14719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:15.903765 14719 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:15.916582 14719 webserver.cc:460] Webserver started at http://127.0.198.194:37189/ using document root <none> and password file <none>
I20260807 19:47:15.917629 14719 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:15.917834 14719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:47:15 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:15.957727 14719 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.012s sys 0.008s
I20260807 19:47:15.968300 14746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:15.970504 14719 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260807 19:47:15.970752 14719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "9d95ca7199254483948372e200c4f98d"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-btw8"
server_key: "de88a24adb5d10ae0f0cf55fccb5b0ff"
server_key_iv: "b46552ed9ebffb95178f60e6d935fa0d"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:15.972430 14719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:15.990653 14719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:15.995425 14719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:15.995760 14719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:15.997016 14719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:15.998693 14719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:15.998821 14719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:15.998974 14719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:15.999065 14719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:16.063223 14719 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35785
I20260807 19:47:16.063382 14878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35785 every 8 connection(s)
I20260807 19:47:16.064953 14719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:16.068421 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14719
I20260807 19:47:16.069299 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.198.212:56079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:36925
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:46193
--webserver_port=40595
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:16.110318 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.069105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.194:49121)
0807 19:47:16.080684 (+ 11579us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:16.080748 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:16.083410 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:16.084785 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:16.084801 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:16.085527 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:16.086479 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:16.086508 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.088227 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.088233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:16.090612 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:16.090648 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.091938 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.091949 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:16.092333 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.093204 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:16.093242 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:16.098551 (+ 5309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:16.104355 (+ 5804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:16.104373 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:16.104391 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:16.104927 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:16.105453 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:16.105457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:16.105460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:16.105764 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:16.106569 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:16.106579 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:16.107111 (+ 532us) client_negotiation.cc:769] Sending connection context
0807 19:47:16.107949 (+ 838us) client_negotiation.cc:240] Negotiation successful
0807 19:47:16.108651 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11176,"thread_start_us":11246,"threads_started":1}
I20260807 19:47:16.111806 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.068341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:49121 (local address 127.0.198.254:46193)
0807 19:47:16.080504 (+ 12163us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:16.080512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:16.081546 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 19:47:16.083995 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:16.084001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:16.084110 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:16.084225 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:16.086750 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.087994 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.090884 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.091634 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.091806 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.098988 (+ 7182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:16.098997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:16.099002 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:16.099058 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:16.104107 (+ 5049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.105113 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.105120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.105124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.105296 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.105933 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.105938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.105941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.106293 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:16.106463 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:16.111016 (+ 4553us) server_negotiation.cc:299] Negotiation successful
0807 19:47:16.111390 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11969,"thread_start_us":13208,"threads_started":1}
I20260807 19:47:16.114449 14879 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:16.114869 14879 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:16.115926 14879 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:16.118244 14507 ts_manager.cc:194] Registered new tserver with Master: 9d95ca7199254483948372e200c4f98d (127.0.198.194:35785)
I20260807 19:47:16.120576 14507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:49121
I20260807 19:47:16.361100 14714 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:16.361579 14714 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:16.363122 14507 ts_manager.cc:194] Registered new tserver with Master: 0c502e14bfb3425590faab588343a35a (127.0.198.193:43631)
I20260807 19:47:16.364645 14507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:56383
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
W20260807 19:47:16.601301 14884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:16.601714 14884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:16.601890 14884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:16.609844 14884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:16.610023 14884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:16.610226 14884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:16.623637 14884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.198.212:56079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:36925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=40595
--tserver_master_addrs=127.0.198.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:16.625515 14884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:16.627462 14884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:16.646248 14889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:16.646279 14890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:16.652787 14884 server_base.cc:1034] running on GCE node
W20260807 19:47:16.653175 14892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:16.654115 14884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:16.655550 14884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:16.656847 14884 hybrid_clock.cc:648] HybridClock initialized: now 1786132036656766 us; error 64 us; skew 500 ppm
I20260807 19:47:16.657300 14884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:16.662084 14884 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:16.664606 14884 webserver.cc:460] Webserver started at http://127.0.198.195:40595/ using document root <none> and password file <none>
I20260807 19:47:16.665642 14884 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:16.665814 14884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:16.695914 14884 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 19:47:16.700071 14901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:16.702102 14884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:47:16.702319 14884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "acbb8c1c10194f15879782368a6df3dd"
format_stamp: "Formatted at 2026-08-07 19:47:13 on dist-test-slave-btw8"
server_key: "f6598c7d13643e784652bd8111f6da40"
server_key_iv: "8dce92efba755913f904309a63377a03"
server_key_version: "kuduclusterkey@0"
I20260807 19:47:16.703861 14884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:16.726250 14884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:16.732509 14884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:16.732900 14884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:16.734377 14884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:16.736773 14884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:16.736914 14884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:16.737046 14884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:16.737141 14884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:16.823706 15021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36925 every 8 connection(s)
I20260807 19:47:16.823979 14884 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36925
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for HTTP/127.0.198.212@KRBTEST.COM
I20260807 19:47:16.826191 14884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:16.826720 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 14884
Aug 07 19:47:16 dist-test-slave-btw8 krb5kdc[13447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:16.918576 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.827925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.0.198.254:46193)
0807 19:47:16.875848 (+ 47923us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:16.875856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:16.884191 (+ 8335us) server_negotiation.cc:407] Connection header received
0807 19:47:16.884425 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:16.884433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:16.884526 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:16.884645 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:16.889742 (+ 5097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.891123 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.898418 (+ 7295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.899240 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.899449 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.903343 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:16.903371 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:16.903376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:16.903443 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:16.909373 (+ 5930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.911056 (+ 1683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.911063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.911068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.911333 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.915496 (+ 4163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.915503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.915507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.915953 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:16.916256 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:16.917897 (+ 1641us) server_negotiation.cc:299] Negotiation successful
0807 19:47:16.918181 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47755,"thread_start_us":48857,"threads_started":1}
I20260807 19:47:16.937455 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.876996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:54685 (local address 127.0.198.254:46193)
0807 19:47:16.881943 (+ 4947us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:16.881956 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:16.881991 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:47:16.882091 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:16.882101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:16.882281 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:16.882507 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:16.904068 (+ 21561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.905817 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.910806 (+ 4989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.911664 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.911784 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.923607 (+ 11823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:16.923615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:16.923620 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:16.923680 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:16.927580 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.930535 (+ 2955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.930542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.930545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.930797 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.934722 (+ 3925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.934729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.934734 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.935190 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:16.935402 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:16.936701 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0807 19:47:16.937006 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4694,"thread_start_us":5715,"threads_started":1}
I20260807 19:47:16.940591 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.837592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:46193 (local address 127.0.198.195:54685)
0807 19:47:16.877950 (+ 40358us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:16.878012 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:16.880476 (+ 2464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:16.897966 (+ 17490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:16.897983 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:16.898834 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:16.903761 (+ 4927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:16.903797 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.906362 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.906369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:16.908951 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:16.908989 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.912617 (+ 3628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.912625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:16.913012 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.913949 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:16.913982 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:16.922420 (+ 8438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:16.927820 (+ 5400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:16.927833 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:16.927857 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:16.928699 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:16.931079 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:16.931085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:16.931089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:16.931551 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:16.935534 (+ 3983us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:16.935544 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:16.936138 (+ 594us) client_negotiation.cc:769] Sending connection context
0807 19:47:16.938097 (+ 1959us) client_negotiation.cc:240] Negotiation successful
0807 19:47:16.938857 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40015,"thread_start_us":39095,"threads_started":1}
I20260807 19:47:16.942948 15022 heartbeater.cc:347] Connected to a master server at 127.0.198.254:46193
I20260807 19:47:16.943413 15022 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:16.944391 15022 heartbeater.cc:511] Master 127.0.198.254:46193 requested a full tablet report, sending...
I20260807 19:47:16.946626 14506 ts_manager.cc:194] Registered new tserver with Master: acbb8c1c10194f15879782368a6df3dd (127.0.198.195:36925)
I20260807 19:47:16.949707 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:16.952345 14506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:54685
I20260807 19:47:17.046577 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.009720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.0.198.254:46193)
0807 19:47:17.009966 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.009974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.023121 (+ 13147us) server_negotiation.cc:407] Connection header received
0807 19:47:17.024083 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.024091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.024210 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.024311 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:17.029186 (+ 4875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.030381 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.033006 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.033704 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.033855 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.038005 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:17.038013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:17.038018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:17.038075 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:17.042406 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.043779 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.043786 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.043790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.043997 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.044559 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.044565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.044569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.045007 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:17.045261 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.045922 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.046208 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:47:17.054690 14506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:17.075156 14506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:17.126045 14879 heartbeater.cc:503] Master 127.0.198.254:46193 was elected leader, sending a full tablet report...
I20260807 19:47:17.133103 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.105681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.0.198.195:36925)
0807 19:47:17.111966 (+ 6285us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.111980 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.121798 (+ 9818us) server_negotiation.cc:407] Connection header received
0807 19:47:17.122195 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.122209 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.122595 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.122827 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:17.123669 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.125886 (+ 2217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.129453 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.130736 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.130826 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.131068 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.132590 (+ 1522us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.132793 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6034,"thread_start_us":9447,"threads_started":1}
I20260807 19:47:17.133985 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.105504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:36925 (local address 127.0.0.1:53158)
0807 19:47:17.117763 (+ 12259us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.121653 (+ 3890us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:17.121921 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:17.122843 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:17.122852 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:17.122886 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:17.123448 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.123470 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.126182 (+ 2712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.126191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.129246 (+ 3055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.129268 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.132071 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.132078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.132285 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.132342 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 19:47:17.133421 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0807 19:47:17.133645 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11974,"thread_start_us":15721,"threads_started":1}
I20260807 19:47:17.138132 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.108263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.0.198.194:35785)
0807 19:47:17.117121 (+ 8858us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.117132 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.121803 (+ 4671us) server_negotiation.cc:407] Connection header received
0807 19:47:17.126928 (+ 5125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.126943 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.127447 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.127696 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:17.128784 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.130332 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.133589 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.135285 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.135466 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.135726 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.137591 (+ 1865us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.137783 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8671,"thread_start_us":8522,"threads_started":1}
I20260807 19:47:17.139163 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.107633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35785 (local address 127.0.0.1:51628)
0807 19:47:17.121563 (+ 13930us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.121632 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:17.126543 (+ 4911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:17.128084 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:17.128089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:17.128119 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:17.128524 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.128536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.130535 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.130541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.133376 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.133397 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.137131 (+ 3734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.137140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.137350 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.137386 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:47:17.138633 (+ 1247us) client_negotiation.cc:240] Negotiation successful
0807 19:47:17.138811 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13777,"thread_start_us":9700,"threads_started":1}
I20260807 19:47:17.138725 14954 tablet_service.cc:1467] Processing CreateTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 (DEFAULT_TABLE table=test-table [id=3025123eb07343bab68bf66587723bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:17.140535 14954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be10fb56e834f83a7d1469be8fd9027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:17.143858 14804 tablet_service.cc:1467] Processing CreateTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 (DEFAULT_TABLE table=test-table [id=3025123eb07343bab68bf66587723bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:17.146446 14804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be10fb56e834f83a7d1469be8fd9027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:17.156183 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.118428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51714)
0807 19:47:17.127497 (+ 9069us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.127537 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:17.130040 (+ 2503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:17.143607 (+ 13567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:17.143614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:17.143650 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:17.144009 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.144023 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.149608 (+ 5585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.149617 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.152227 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.152256 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.155183 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.155195 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.155462 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.155512 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:47:17.155601 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:47:17.155794 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8846,"thread_start_us":9259,"threads_started":1}
I20260807 19:47:17.157614 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.118807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.0.198.193:43631)
0807 19:47:17.142327 (+ 23520us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.142336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.142370 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:47:17.142558 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.142567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.142896 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.143163 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:17.144321 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.146228 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.152564 (+ 6336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.154855 (+ 2291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.156668 (+ 1813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.156956 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.157055 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.157251 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23340,"thread_start_us":23157,"threads_started":1}
I20260807 19:47:17.162281 14649 tablet_service.cc:1467] Processing CreateTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 (DEFAULT_TABLE table=test-table [id=3025123eb07343bab68bf66587723bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:17.164199 14649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be10fb56e834f83a7d1469be8fd9027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:17.177400 15043 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Bootstrap starting.
I20260807 19:47:17.182909 15043 tablet_bootstrap.cc:654] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:17.184176 15044 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Bootstrap starting.
I20260807 19:47:17.184725 15043 log.cc:824] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:17.190474 15044 tablet_bootstrap.cc:654] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:17.192060 15044 log.cc:824] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:17.197479 15044 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: No bootstrap required, opened a new log
I20260807 19:47:17.197832 15044 ts_tablet_manager.cc:1397] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260807 19:47:17.200322 15046 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Bootstrap starting.
I20260807 19:47:17.210026 15046 tablet_bootstrap.cc:654] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:17.212014 15043 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: No bootstrap required, opened a new log
I20260807 19:47:17.212383 15043 ts_tablet_manager.cc:1397] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Time spent bootstrapping tablet: real 0.035s user 0.016s sys 0.015s
I20260807 19:47:17.212931 15046 log.cc:824] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:17.221220 15044 raft_consensus.cc:348] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.221803 15044 raft_consensus.cc:374] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:17.221908 15044 raft_consensus.cc:729] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d95ca7199254483948372e200c4f98d, State: Initialized, Role: FOLLOWER
I20260807 19:47:17.222935 15044 consensus_queue.cc:260] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.224187 15043 raft_consensus.cc:348] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.224740 15043 raft_consensus.cc:374] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:17.224858 15043 raft_consensus.cc:729] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acbb8c1c10194f15879782368a6df3dd, State: Initialized, Role: FOLLOWER
I20260807 19:47:17.225662 15043 consensus_queue.cc:260] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.226967 15046 tablet_bootstrap.cc:492] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: No bootstrap required, opened a new log
I20260807 19:47:17.227407 15046 ts_tablet_manager.cc:1397] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.000s
I20260807 19:47:17.239704 15044 ts_tablet_manager.cc:1428] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Time spent starting tablet: real 0.042s user 0.022s sys 0.003s
I20260807 19:47:17.239465 15046 raft_consensus.cc:348] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.240293 15046 raft_consensus.cc:374] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:17.240471 15046 raft_consensus.cc:729] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c502e14bfb3425590faab588343a35a, State: Initialized, Role: FOLLOWER
I20260807 19:47:17.241263 15046 consensus_queue.cc:260] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.257825 15022 heartbeater.cc:503] Master 127.0.198.254:46193 was elected leader, sending a full tablet report...
I20260807 19:47:17.261056 14714 heartbeater.cc:503] Master 127.0.198.254:46193 was elected leader, sending a full tablet report...
I20260807 19:47:17.262540 15043 ts_tablet_manager.cc:1428] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Time spent starting tablet: real 0.050s user 0.025s sys 0.018s
I20260807 19:47:17.263481 15046 ts_tablet_manager.cc:1428] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Time spent starting tablet: real 0.036s user 0.014s sys 0.020s
W20260807 19:47:17.317610 14880 tablet.cc:2369] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:47:17.320210 14715 tablet.cc:2369] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:47:17.378207 15025 tablet.cc:2369] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:17.492226 15051 raft_consensus.cc:482] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:17.492607 15051 raft_consensus.cc:504] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.494603 15051 leader_election.cc:288] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d95ca7199254483948372e200c4f98d (127.0.198.194:35785), 0c502e14bfb3425590faab588343a35a (127.0.198.193:43631)
I20260807 19:47:17.504894 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.495170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:36609 (local address 127.0.198.194:35785)
0807 19:47:17.495453 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.495461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.495913 (+ 452us) server_negotiation.cc:407] Connection header received
0807 19:47:17.496177 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.496186 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.496320 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.496412 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:17.497829 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.499497 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.502460 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.503779 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.503995 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.504076 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.504447 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.504606 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:47:17.505209 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.494918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35785 (local address 127.0.198.195:36609)
0807 19:47:17.495778 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.495802 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:17.495962 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:17.496942 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:17.496952 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:17.497001 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:17.497563 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.497575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.499734 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.499738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.502247 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.502264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.503965 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.503972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.504230 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.504286 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 19:47:17.504390 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:47:17.504550 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":1,"thread_start_us":563,"threads_started":1}
I20260807 19:47:17.506597 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.496207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:53887 (local address 127.0.198.193:43631)
0807 19:47:17.496411 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.496417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.499932 (+ 3515us) server_negotiation.cc:407] Connection header received
0807 19:47:17.500072 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.500078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.500187 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.500293 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:17.501078 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.502273 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.504355 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.505547 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.505723 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.505791 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.506205 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.506348 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:47:17.506765 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.496014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.198.195:53887)
0807 19:47:17.499777 (+ 3763us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.499802 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:17.499944 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:17.500339 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:17.500345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:17.500381 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:17.500861 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.500873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.502491 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.502495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.504162 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:17.504178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.505761 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.505767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:17.505976 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.506027 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 19:47:17.506156 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:47:17.506315 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3613,"mutex_wait_us":36,"thread_start_us":3396,"threads_started":1}
I20260807 19:47:17.506518 14828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be10fb56e834f83a7d1469be8fd9027" candidate_uuid: "acbb8c1c10194f15879782368a6df3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d95ca7199254483948372e200c4f98d" is_pre_election: true
I20260807 19:47:17.507203 14828 raft_consensus.cc:2454] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acbb8c1c10194f15879782368a6df3dd in term 0.
I20260807 19:47:17.508119 14902 leader_election.cc:302] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d95ca7199254483948372e200c4f98d, acbb8c1c10194f15879782368a6df3dd; no voters:
I20260807 19:47:17.508091 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be10fb56e834f83a7d1469be8fd9027" candidate_uuid: "acbb8c1c10194f15879782368a6df3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c502e14bfb3425590faab588343a35a" is_pre_election: true
I20260807 19:47:17.508623 15051 raft_consensus.cc:2781] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:17.508795 15051 raft_consensus.cc:482] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:17.508713 14669 raft_consensus.cc:2454] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acbb8c1c10194f15879782368a6df3dd in term 0.
I20260807 19:47:17.508895 15051 raft_consensus.cc:3037] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:17.512102 15051 raft_consensus.cc:504] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.512800 15051 leader_election.cc:288] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [CANDIDATE]: Term 1 election: Requested vote from peers 9d95ca7199254483948372e200c4f98d (127.0.198.194:35785), 0c502e14bfb3425590faab588343a35a (127.0.198.193:43631)
I20260807 19:47:17.513388 14828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be10fb56e834f83a7d1469be8fd9027" candidate_uuid: "acbb8c1c10194f15879782368a6df3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d95ca7199254483948372e200c4f98d"
I20260807 19:47:17.513388 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be10fb56e834f83a7d1469be8fd9027" candidate_uuid: "acbb8c1c10194f15879782368a6df3dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c502e14bfb3425590faab588343a35a"
I20260807 19:47:17.513624 14669 raft_consensus.cc:3037] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:17.513624 14828 raft_consensus.cc:3037] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:17.517056 14669 raft_consensus.cc:2454] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acbb8c1c10194f15879782368a6df3dd in term 1.
I20260807 19:47:17.517056 14828 raft_consensus.cc:2454] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acbb8c1c10194f15879782368a6df3dd in term 1.
I20260807 19:47:17.517688 14902 leader_election.cc:302] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d95ca7199254483948372e200c4f98d, acbb8c1c10194f15879782368a6df3dd; no voters:
I20260807 19:47:17.518074 15051 raft_consensus.cc:2781] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:17.518898 15051 raft_consensus.cc:686] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 1 LEADER]: Becoming Leader. State: Replica: acbb8c1c10194f15879782368a6df3dd, State: Running, Role: LEADER
I20260807 19:47:17.519518 15051 consensus_queue.cc:237] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } }
I20260807 19:47:17.525521 14507 catalog_manager.cc:5495] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to acbb8c1c10194f15879782368a6df3dd (127.0.198.195). New cstate: current_term: 1 leader_uuid: "acbb8c1c10194f15879782368a6df3dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acbb8c1c10194f15879782368a6df3dd" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:17.602259 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.596088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.0.198.195:36925)
0807 19:47:17.596358 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.596368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.596724 (+ 356us) server_negotiation.cc:407] Connection header received
0807 19:47:17.597028 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.597032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.597115 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.597291 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:17.597706 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.598444 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.599967 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.600376 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.600528 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.601067 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:17.601615 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:17.601789 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.601888 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.601976 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:17.611369 14828 raft_consensus.cc:1261] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 1 FOLLOWER]: Refusing update from remote peer acbb8c1c10194f15879782368a6df3dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:17.611369 14669 raft_consensus.cc:1261] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 1 FOLLOWER]: Refusing update from remote peer acbb8c1c10194f15879782368a6df3dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:17.612462 15062 consensus_queue.cc:1035] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c502e14bfb3425590faab588343a35a" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:17.612903 15051 consensus_queue.cc:1035] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d95ca7199254483948372e200c4f98d" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:17.626652 15067 mvcc.cc:205] Tried to move back new op lower bound from 7315996826044219392 to 7315996825684733952. Current Snapshot: MvccSnapshot[applied={T|T < 7315996826044219392}]
I20260807 19:47:17.642751 14507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:17.643117 14507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:17.647486 14507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddf587eaa6784ce88c3b247dd9cdc3a6: Sending DeleteTablet for 3 replicas of tablet 5be10fb56e834f83a7d1469be8fd9027
I20260807 19:47:17.649115 14649 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:17 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:51714
I20260807 19:47:17.649211 14804 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:17 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:51628
I20260807 19:47:17.649286 14954 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be10fb56e834f83a7d1469be8fd9027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:17 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:53158
I20260807 19:47:17.653647 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14547
I20260807 19:47:17.660131 15076 tablet_replica.cc:330] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: stopping tablet replica
I20260807 19:47:17.660805 15076 raft_consensus.cc:2229] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:17.661535 15076 raft_consensus.cc:2258] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:17.665798 15076 ts_tablet_manager.cc:1905] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:17.672082 15075 tablet_replica.cc:330] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: stopping tablet replica
I20260807 19:47:17.672729 15075 raft_consensus.cc:2229] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:17.673411 15075 raft_consensus.cc:2258] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:17.674860 15077 tablet_replica.cc:330] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: stopping tablet replica
I20260807 19:47:17.675302 15077 raft_consensus.cc:2229] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:47:17.675972 15075 ts_tablet_manager.cc:1905] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:17.676338 15077 raft_consensus.cc:2258] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:17.678900 15077 ts_tablet_manager.cc:1905] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:17.684242 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.655484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51726 (local address 127.0.198.193:43631)
0807 19:47:17.655737 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.655747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.662823 (+ 7076us) server_negotiation.cc:407] Connection header received
0807 19:47:17.663069 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.663078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.663216 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.663344 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:17.666093 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.667797 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.670558 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.671597 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.671724 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.675506 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:17.675520 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:17.675540 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:17.675587 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:17.680294 (+ 4707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.681271 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.681280 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.681283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.681470 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.682027 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.682032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.682035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.682519 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:17.682835 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.683515 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.683811 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:47:17.685174 15075 ts_tablet_manager.cc:1918] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:17.685454 15075 log.cc:1196] T 5be10fb56e834f83a7d1469be8fd9027 P 0c502e14bfb3425590faab588343a35a: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/5be10fb56e834f83a7d1469be8fd9027
I20260807 19:47:17.685861 14709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:17.686902 15077 ts_tablet_manager.cc:1918] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:17.687119 15077 log.cc:1196] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/5be10fb56e834f83a7d1469be8fd9027
I20260807 19:47:17.687764 15077 ts_tablet_manager.cc:1939] T 5be10fb56e834f83a7d1469be8fd9027 P acbb8c1c10194f15879782368a6df3dd: Deleting consensus metadata
I20260807 19:47:17.689977 14491 catalog_manager.cc:4841] TS acbb8c1c10194f15879782368a6df3dd (127.0.198.195:36925): tablet 5be10fb56e834f83a7d1469be8fd9027 (table test-table [id=3025123eb07343bab68bf66587723bf7]) successfully deleted
I20260807 19:47:17.690220 15076 ts_tablet_manager.cc:1918] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:17.690407 15076 log.cc:1196] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/5be10fb56e834f83a7d1469be8fd9027
I20260807 19:47:17.691031 15076 ts_tablet_manager.cc:1939] T 5be10fb56e834f83a7d1469be8fd9027 P 9d95ca7199254483948372e200c4f98d: Deleting consensus metadata
I20260807 19:47:17.693730 14488 catalog_manager.cc:4841] TS 9d95ca7199254483948372e200c4f98d (127.0.198.194:35785): tablet 5be10fb56e834f83a7d1469be8fd9027 (table test-table [id=3025123eb07343bab68bf66587723bf7]) successfully deleted
I20260807 19:47:17.785264 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14547
W20260807 19:47:17.805176 14488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:43631 (error 108)
I20260807 19:47:17.805348 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14719
I20260807 19:47:17.809701 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.806980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51734)
0807 19:47:17.809309 (+ 2329us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.809456 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2187,"thread_start_us":2066,"threads_started":1}
W20260807 19:47:17.810050 14488 catalog_manager.cc:4584] TS 0c502e14bfb3425590faab588343a35a (127.0.198.193:43631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5be10fb56e834f83a7d1469be8fd9027: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
I20260807 19:47:17.821779 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.806267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.0.198.194:35785)
0807 19:47:17.806453 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.806458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.806607 (+ 149us) server_negotiation.cc:407] Connection header received
0807 19:47:17.806807 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.806814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.806913 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.807027 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:17.808973 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.809875 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.811551 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.812047 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.812197 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.815018 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:17.815033 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:17.815049 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:17.815087 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:17.818361 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.819265 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.819282 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.819286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.819625 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.820046 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.820051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.820055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.820455 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:17.820814 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.821137 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.821386 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:47:17.824221 14874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:17.864080 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.863502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51746)
0807 19:47:17.863747 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.863889 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:47:17.865423 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.864945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51756)
0807 19:47:17.865142 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.865247 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:47:17.919924 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14719
I20260807 19:47:17.926374 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.925615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51772)
0807 19:47:17.925935 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.926084 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 19:47:17.927780 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.927266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51786)
0807 19:47:17.927484 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:17.927586 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:47:17.942094 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14884
I20260807 19:47:17.960296 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:17.942967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.0.198.195:36925)
0807 19:47:17.943143 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:17.943148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:17.943196 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:47:17.943497 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:17.943503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:17.943595 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:17.943721 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:17.945578 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.946525 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.948624 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:17.949116 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:17.949232 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:17.953170 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:17.953184 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:17.953200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:17.953240 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:17.956785 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.957891 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.957900 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.957902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.958138 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:17.958661 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:17.958665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:17.958667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:17.959064 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:17.959515 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:17.959739 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:47:17.959954 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:47:17.961187 15010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:18.008553 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:18.007939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51802)
0807 19:47:18.008231 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:18.008357 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:47:18.009877 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:18.009353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51804)
0807 19:47:18.009583 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:18.009676 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 19:47:18.090521 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14884
I20260807 19:47:18.113231 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 14470
I20260807 19:47:18.114559 14539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:18.234855 15081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:18.234010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43631 (local address 127.0.0.1:51820)
0807 19:47:18.234382 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:18.234524 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260807 19:47:18.235031 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 14470
I20260807 19:47:18.265657 795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:47:18.612843 795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:47:18.613091 795 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:47:18.960361 795 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:47:18.960 UTC [13477] LOG: received smart shutdown request
2026-08-07 19:47:18.964 UTC [13477] LOG: background worker "logical replication launcher" (PID 13485) exited with exit code 1
2026-08-07 19:47:18.965 UTC [13480] LOG: shutting down
2026-08-07 19:47:19.164 UTC [13477] LOG: database system is shut down
2026-08-07T19:47:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (60988 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:19 dist-test-slave-btw8 krb5kdc[15093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:19 dist-test-slave-btw8 krb5kdc[15093](info): set up 2 sockets
Aug 07 19:47:19 dist-test-slave-btw8 krb5kdc[15093](info): commencing operation
krb5kdc: starting...
W20260807 19:47:20.500247 795 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:47:20 dist-test-slave-btw8 krb5kdc[15093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:47:20.828404 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40209
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:21.189703 15109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:21.190062 15109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:21.190158 15109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:21.197791 15109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:21.197919 15109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:21.197973 15109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:21.198035 15109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:21.198078 15109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:21.208503 15109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:21.209967 15109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:21.211433 15109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:21.224597 15115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:21.224721 15117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:21.224730 15114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:21.225284 15109 server_base.cc:1034] running on GCE node
I20260807 19:47:21.226243 15109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:21.227994 15109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:21.229231 15109 hybrid_clock.cc:648] HybridClock initialized: now 1786132041229181 us; error 40 us; skew 500 ppm
I20260807 19:47:21.229694 15109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:21 dist-test-slave-btw8 krb5kdc[15093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:21.233266 15109 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:21.235047 15109 webserver.cc:460] Webserver started at http://127.0.198.254:43311/ using document root <none> and password file <none>
I20260807 19:47:21.235919 15109 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:21.236089 15109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:21.236441 15109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:21.240021 15109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "bf8bff364c124f3b80aa2e7bdf0d396b"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "ec1f080ba4019baf8e883bb3e909afe6"
server_key_iv: "4589a279fb138759bdebe223dea100f2"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.240985 15109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "bf8bff364c124f3b80aa2e7bdf0d396b"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "ec1f080ba4019baf8e883bb3e909afe6"
server_key_iv: "4589a279fb138759bdebe223dea100f2"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.247025 15109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:47:21.251144 15124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:21.252840 15109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:21.253132 15109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "bf8bff364c124f3b80aa2e7bdf0d396b"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "ec1f080ba4019baf8e883bb3e909afe6"
server_key_iv: "4589a279fb138759bdebe223dea100f2"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.253407 15109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:21.272843 15109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:21.277926 15109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:21.278260 15109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:21.299192 15109 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:40209
I20260807 19:47:21.299224 15178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:40209 every 8 connection(s)
I20260807 19:47:21.300840 15109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:21.302937 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15109
I20260807 19:47:21.303192 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:47:21.303980 795 external_mini_cluster.cc:1428] Setting key c63522218e2bb185a4a21199c32385cc
I20260807 19:47:21.305181 15179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:21.312506 15179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:21 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.253@KRBTEST.COM" created.
I20260807 19:47:21.340677 15185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.317448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.0.198.254:40209)
0807 19:47:21.318235 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:21.318249 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:21.318311 (+ 62us) server_negotiation.cc:407] Connection header received
0807 19:47:21.319665 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:21.319677 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:21.320033 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:21.320254 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:21.321789 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.323886 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.325623 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.326711 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.326930 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.331226 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:21.331243 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:21.331258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:21.331306 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:21.334654 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.335939 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.335954 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.335957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.336179 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.336766 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.336770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.336773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.337432 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:21.337620 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:21.338393 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 19:47:21.338862 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":570,"thread_start_us":414,"threads_started":1}
I20260807 19:47:21.340704 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.316225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40209 (local address 127.0.0.1:50152)
0807 19:47:21.317144 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.317187 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:21.319175 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:21.320335 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:21.320341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:21.321111 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:21.321540 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.321562 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.324092 (+ 2530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.324096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.325434 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.325451 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.326987 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.326993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.327172 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.327993 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:21.328017 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:21.330860 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:21.334837 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.334853 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.334870 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.335554 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.336348 (+ 794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.336352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.336354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.336540 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.337634 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:21.337642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:21.337882 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 19:47:21.338230 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 19:47:21.339009 (+ 779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":107,"thread_start_us":366,"threads_started":1}
I20260807 19:47:21.346218 15179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:21.347736 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.347140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34322)
0807 19:47:21.347431 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.347579 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 19:47:21.348070 15125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
I20260807 19:47:21.350116 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.349592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34332)
0807 19:47:21.349856 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.349947 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260807 19:47:21.350375 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:42759: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
I20260807 19:47:21.379463 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:42759
--webserver_interface=127.0.198.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:21.399835 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } attempt: 1
I20260807 19:47:21.401372 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.400769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34344)
0807 19:47:21.401016 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.401191 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 19:47:21.402557 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.402135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34356)
0807 19:47:21.402312 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.402406 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:47:21.402767 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:42759: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
I20260807 19:47:21.471131 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } attempt: 2
I20260807 19:47:21.472710 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.472033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34370)
0807 19:47:21.472385 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.472510 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 19:47:21.473994 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.473533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34376)
0807 19:47:21.473704 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.473814 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:47:21.474242 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:42759: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
I20260807 19:47:21.565557 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } attempt: 3
I20260807 19:47:21.566927 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.566386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34388)
0807 19:47:21.566635 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.566765 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 19:47:21.568079 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.567662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34396)
0807 19:47:21.567819 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.567906 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:47:21.568295 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:42759: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
I20260807 19:47:21.711587 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } attempt: 4
I20260807 19:47:21.713167 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.712378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34402)
0807 19:47:21.712785 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.712929 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260807 19:47:21.714612 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.714016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34416)
0807 19:47:21.714223 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.714411 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:47:21.714900 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:42759: Network error: Client connection negotiation failed: client connection to 127.0.198.253:42759: connect: Connection refused (error 111)
W20260807 19:47:21.750166 15188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:21.750515 15188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:21.750631 15188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:21.759073 15188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:21.759218 15188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:21.759295 15188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:21.759438 15188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:21.759528 15188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:21.770206 15188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:42759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.198.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:21.771813 15188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:21.773212 15188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:21.786720 15194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:21.786966 15196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:21.786803 15193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:21.788183 15188 server_base.cc:1034] running on GCE node
I20260807 19:47:21.789160 15188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:21.790831 15188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:21.792104 15188 hybrid_clock.cc:648] HybridClock initialized: now 1786132041792042 us; error 60 us; skew 500 ppm
I20260807 19:47:21.792514 15188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:21 dist-test-slave-btw8 krb5kdc[15093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:21.796259 15188 init.cc:377] Logged in from keytab as kudu/127.0.198.253@KRBTEST.COM (short username kudu)
I20260807 19:47:21.798298 15188 webserver.cc:460] Webserver started at http://127.0.198.253:35935/ using document root <none> and password file <none>
I20260807 19:47:21.799078 15188 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:21.799244 15188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:21.799645 15188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:21.802958 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data/instance:
uuid: "5ed2cb7921c04ce6a66dce8e86d24058"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "a010ca4a24ba82cea30ca776fc42917d"
server_key_iv: "f485edb14d01cd33b6c018746454720c"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.803871 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal/instance:
uuid: "5ed2cb7921c04ce6a66dce8e86d24058"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "a010ca4a24ba82cea30ca776fc42917d"
server_key_iv: "f485edb14d01cd33b6c018746454720c"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.809571 15188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:47:21.813524 15203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:21.815088 15188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:47:21.815294 15188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "5ed2cb7921c04ce6a66dce8e86d24058"
format_stamp: "Formatted at 2026-08-07 19:47:21 on dist-test-slave-btw8"
server_key: "a010ca4a24ba82cea30ca776fc42917d"
server_key_iv: "f485edb14d01cd33b6c018746454720c"
server_key_version: "encryptionkey@0"
I20260807 19:47:21.815582 15188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:21.834690 15188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:21.840505 15188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:21.840893 15188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:21.863090 15188 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.253:42759
I20260807 19:47:21.863199 15256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.253:42759 every 8 connection(s)
I20260807 19:47:21.864733 15188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
I20260807 19:47:21.869091 15258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:21.873522 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15188
I20260807 19:47:21.873755 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-1/wal/instance
I20260807 19:47:21.874266 795 external_mini_cluster.cc:1428] Setting key 8a3ae0600e90a8e489268d5cd668bb57
I20260807 19:47:21.876986 15258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:21 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.252@KRBTEST.COM; defaulting to no policy
I20260807 19:47:21.908712 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.880148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.0.198.254:40209)
0807 19:47:21.880729 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:21.880737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:21.881861 (+ 1124us) server_negotiation.cc:407] Connection header received
0807 19:47:21.884040 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:21.884049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:21.884157 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:21.884307 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:21.886423 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.887645 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.891713 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.892249 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.892392 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.897886 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:21.897892 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:21.897895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:21.897948 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:21.902508 (+ 4560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.904487 (+ 1979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.904495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.904499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.904804 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.905583 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.905590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.905593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.906183 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:21.906383 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:21.907794 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0807 19:47:21.908141 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":274,"threads_started":1}
I20260807 19:47:21.909649 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.880751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40209 (local address 127.0.0.1:50166)
0807 19:47:21.881644 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.881712 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:21.883718 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:21.884577 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:21.884586 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:21.885489 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:21.886167 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.886190 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.888235 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.888244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.891317 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.891388 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.892585 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.892598 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.893030 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.894124 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:21.894167 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:21.897545 (+ 3378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:21.902981 (+ 5436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.903000 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.903023 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.904153 (+ 1130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.905009 (+ 856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.905019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.905025 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.905374 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.906390 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:21.906431 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:21.907080 (+ 649us) client_negotiation.cc:769] Sending connection context
0807 19:47:21.907529 (+ 449us) client_negotiation.cc:240] Negotiation successful
0807 19:47:21.908180 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":380,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.252@KRBTEST.COM" created.
I20260807 19:47:21.911530 15258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:21 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:47:21.927750 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.912200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34418)
0807 19:47:21.912424 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.912444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:21.912540 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:21.913584 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:21.913589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:21.914081 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:21.914231 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.914240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.915707 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.915712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.916562 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:21.916574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.917473 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.917479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:21.917698 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.918376 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:21.918394 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:21.921163 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:21.925002 (+ 3839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.925009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.925013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.925428 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.925862 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:21.925869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:21.925877 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:21.926014 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:21.926662 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:21.926669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:21.926827 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 19:47:21.926970 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 19:47:21.927306 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:47:21.927846 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.912539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.0.198.253:42759)
0807 19:47:21.913158 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:21.913165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:21.913191 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:47:21.913239 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:21.913244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:21.913428 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:21.913592 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:21.914394 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.915568 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.916732 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:21.917295 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:21.917458 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:21.921416 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:21.921429 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:21.921443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:21.921482 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:21.924823 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.925582 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.925594 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.925602 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.925732 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:21.926139 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:21.926145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:21.926148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:21.926526 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:21.926773 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:21.927158 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:47:21.927465 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":476,"thread_start_us":333,"threads_started":1}
I20260807 19:47:21.930898 15258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:21.932698 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.932024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52450)
0807 19:47:21.932288 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.932458 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 19:47:21.933097 15206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:21.935111 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.934584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52456)
0807 19:47:21.934808 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.934934 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 19:47:21.935523 15258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
I20260807 19:47:21.952255 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:39835
--webserver_interface=127.0.198.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:21.984953 15258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 1
I20260807 19:47:21.986253 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.985784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52466)
0807 19:47:21.985993 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.986103 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:47:21.987339 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:21.986973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52474)
0807 19:47:21.987152 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:21.987223 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:47:21.987599 15258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.014256 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } attempt: 5
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:47:22.030778 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.015135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34426)
0807 19:47:22.015410 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.015436 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.015584 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.016024 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.016029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.016478 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.016683 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.016693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.017710 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.017716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.018575 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.018586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.019461 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.019471 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.019728 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.020457 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.020481 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.023211 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.027891 (+ 4680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.027896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.027899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.028545 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.029060 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.029064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.029067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.029221 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.029951 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.029955 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.030102 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.030227 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.030519 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:47:22.030948 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.015251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.0.198.253:42759)
0807 19:47:22.015494 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.015501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.015525 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:47:22.015805 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.015810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.015878 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.015989 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.016853 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.017563 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.018742 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.019230 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.019428 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.023523 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.023531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.023536 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.023586 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.027665 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.028715 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.028722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.028726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.028929 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.029391 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.029396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.029400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.029812 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.029997 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.030377 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.030652 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 19:47:22.032042 15179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:22.033224 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.032817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52486)
0807 19:47:22.032997 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.033086 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 19:47:22.034303 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.033970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52498)
0807 19:47:22.034117 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.034203 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:47:22.034544 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.055838 15258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 2
I20260807 19:47:22.057267 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.056689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52512)
0807 19:47:22.056973 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.057098 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:47:22.058353 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.057994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52524)
0807 19:47:22.058153 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.058233 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 19:47:22.058563 15258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.085826 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 1
I20260807 19:47:22.087287 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.086686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52540)
0807 19:47:22.086991 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.087102 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 19:47:22.088637 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.088126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52556)
0807 19:47:22.088337 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.088457 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:47:22.088945 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.149811 15258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 3
I20260807 19:47:22.151084 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.150538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52568)
0807 19:47:22.150817 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.150918 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:47:22.152446 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.151979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52572)
0807 19:47:22.152244 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.152326 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260807 19:47:22.152733 15258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.157186 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 2
I20260807 19:47:22.158452 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.157825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52584)
0807 19:47:22.158122 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.158240 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:47:22.159801 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.159314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52598)
0807 19:47:22.159542 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.159630 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260807 19:47:22.160079 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.266360 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 3
I20260807 19:47:22.268002 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.267251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52606)
0807 19:47:22.267603 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.267732 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 19:47:22.269418 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.268910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52610)
0807 19:47:22.269199 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.269290 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260807 19:47:22.269676 15179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
I20260807 19:47:22.296007 15258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 4
I20260807 19:47:22.297331 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.296778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52618)
0807 19:47:22.297065 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.297160 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 19:47:22.298687 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.298122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52624)
0807 19:47:22.298409 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.298481 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260807 19:47:22.298959 15258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:39835: Network error: Client connection negotiation failed: client connection to 127.0.198.252:39835: connect: Connection refused (error 111)
W20260807 19:47:22.335814 15267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:22.336166 15267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:22.336267 15267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:22.344379 15267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:22.344528 15267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:22.344578 15267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:22.344643 15267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:22.344689 15267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:22.355455 15267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:40209,127.0.198.253:42759,127.0.198.252:39835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:39835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.198.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:22.356856 15267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:22.358232 15267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:22.371183 15275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:22.371088 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:22.371047 15272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:22.372393 15267 server_base.cc:1034] running on GCE node
I20260807 19:47:22.373229 15267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:22.374742 15267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:22.375946 15267 hybrid_clock.cc:648] HybridClock initialized: now 1786132042375888 us; error 56 us; skew 500 ppm
I20260807 19:47:22.376324 15267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:22.379742 15267 init.cc:377] Logged in from keytab as kudu/127.0.198.252@KRBTEST.COM (short username kudu)
I20260807 19:47:22.381304 15267 webserver.cc:460] Webserver started at http://127.0.198.252:39673/ using document root <none> and password file <none>
I20260807 19:47:22.382014 15267 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:22.382120 15267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:22.382382 15267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:22.385836 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data/instance:
uuid: "b89c9cfe8db8429db0398c56aa233383"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-btw8"
server_key: "a45e3fa21e4436b0ab439cc3e0c23211"
server_key_iv: "7f98e4597bcc8bd21a28098d49358d8a"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.386667 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal/instance:
uuid: "b89c9cfe8db8429db0398c56aa233383"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-btw8"
server_key: "a45e3fa21e4436b0ab439cc3e0c23211"
server_key_iv: "7f98e4597bcc8bd21a28098d49358d8a"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.392457 15267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:47:22.396154 15283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:22.397644 15267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:22.397894 15267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "b89c9cfe8db8429db0398c56aa233383"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-btw8"
server_key: "a45e3fa21e4436b0ab439cc3e0c23211"
server_key_iv: "7f98e4597bcc8bd21a28098d49358d8a"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.398138 15267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:22.420354 15267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:22.425134 15267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:22.425396 15267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:22.445453 15267 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.252:39835
I20260807 19:47:22.445493 15337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.252:39835 every 8 connection(s)
I20260807 19:47:22.446888 15179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 4
I20260807 19:47:22.447814 15267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
I20260807 19:47:22.452911 15338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:22.455458 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15267
I20260807 19:47:22.455683 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131811070045-795-0/minicluster-data/master-2/wal/instance
I20260807 19:47:22.456472 795 external_mini_cluster.cc:1428] Setting key 8e741588346e1c9a8169b6e9cae8183b
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:47:22.465561 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:22.476074 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.447751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52640)
0807 19:47:22.447967 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.447990 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.448122 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.453255 (+ 5133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.453262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.453850 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.454076 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.454091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.457115 (+ 3024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.457123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.458927 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.458956 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.461445 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.461456 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.461783 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.462653 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.462689 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.466560 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.472463 (+ 5903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.472473 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.472477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.472964 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.473523 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.473529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.473533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.473710 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.474902 (+ 1192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.474910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.475088 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.475256 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.475606 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:47:22.477902 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.448725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.0.198.252:39835)
0807 19:47:22.449684 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.449701 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.449752 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:47:22.452191 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.452220 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.452737 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.453310 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.454686 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.456884 (+ 2198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.459497 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.461158 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.462376 (+ 1218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.466953 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.466963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.466982 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.467024 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.472289 (+ 5265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.473121 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.473131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.473140 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.473350 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.473843 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.473848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.473852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.474715 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.474933 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.475651 (+ 718us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.476228 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":438,"threads_started":1}
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:47:22.486261 15179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: Bootstrap starting.
I20260807 19:47:22.489485 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.469203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.0.198.254:40209)
0807 19:47:22.471004 (+ 1801us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.471010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.471028 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:47:22.471085 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.471089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.471144 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.471292 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.473030 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.474089 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.476013 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.476736 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.476885 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.481102 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.481109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.481114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.481157 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.485573 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.486732 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.486740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.486745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.486930 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.487472 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.487477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.487481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.487895 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.488086 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.488736 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.488987 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1629,"thread_start_us":1646,"threads_started":1}
I20260807 19:47:22.489611 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.469114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:40209 (local address 127.0.0.1:50178)
0807 19:47:22.470306 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.470353 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.470508 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.471339 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.471373 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.472434 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.472851 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.472864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.474280 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.474283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.475826 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.475851 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.476943 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.476952 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.477120 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.477949 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.477966 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.480812 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.485807 (+ 4995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.485824 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.485831 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.486547 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.487078 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.487084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.487087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.487279 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.488070 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.488080 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.488321 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.488576 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.489005 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":786,"threads_started":1}
I20260807 19:47:22.490873 15179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:22.491493 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:22.492370 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.457652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.0.198.252:39835)
0807 19:47:22.467120 (+ 9468us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.467126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.467151 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:47:22.467205 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.467209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.467294 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.467465 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.471860 (+ 4395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.473310 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.475453 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.476396 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.476525 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.483444 (+ 6919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.483458 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.483462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.483499 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.487313 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.488353 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.488358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.488361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.488525 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.489051 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.489056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.489060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.489467 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.491594 (+ 2127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.491753 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.492027 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9284,"thread_start_us":9047,"threads_started":1}
I20260807 19:47:22.492424 15179 log.cc:824] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:22.496479 15179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: No bootstrap required, opened a new log
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:22.507637 15179 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.508203 15179 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:22.508348 15179 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8bff364c124f3b80aa2e7bdf0d396b, State: Initialized, Role: FOLLOWER
I20260807 19:47:22.509133 15179 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.510321 15347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.510854 15347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.511744 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.493182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.0.198.253:42759)
0807 19:47:22.493385 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.493389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.493405 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:47:22.493456 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.493460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.493519 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.493652 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.494822 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.495886 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.497419 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.498052 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.498191 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.502929 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.502941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.502948 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.503006 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.506499 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.507526 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.507534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.507538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.508014 (+ 476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.508589 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.508594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.508597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.509271 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.509581 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.509977 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.510260 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:47:22.511845 15179 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:22.512733 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.492279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:42759 (local address 127.0.0.1:34434)
0807 19:47:22.492496 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.492524 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.492642 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.493759 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.493765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.494313 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.494526 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.494538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.496057 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.496063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.497243 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.497260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.498455 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.498462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.498681 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.499478 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.499502 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.502594 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.506671 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.506678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.506682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.507337 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.508230 (+ 893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.508236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.508239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.508472 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.509430 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.509437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.509634 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.512041 (+ 2407us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.512413 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:47:22.513906 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:22.515730 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.494852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.0.198.254:40209)
0807 19:47:22.495032 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.495039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.495058 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:22.495119 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.495125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.495244 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.495390 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.497084 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.497978 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.499809 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.500354 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.500517 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.507002 (+ 6485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.507010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.507014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.507059 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.511226 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.512259 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.512265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.512269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.512424 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.512899 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.512906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.512911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.513396 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.513770 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.514142 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.515205 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:47:22.531298 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.514735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52660)
0807 19:47:22.514910 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.514926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.515007 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.515710 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.515714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.516191 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.516432 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.516444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.517662 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.517666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.518994 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.519010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.520392 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.520396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.520511 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.521180 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.521198 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.523762 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.528255 (+ 4493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.528262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.528266 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.528789 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.529218 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.529222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.529225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.529393 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.530197 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.530202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.530373 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.530529 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.530875 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 19:47:22.531739 15130 tablet.cc:2369] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:22.533764 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.515159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.0.198.252:39835)
0807 19:47:22.515318 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.515322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.515336 (+ 14us) server_negotiation.cc:407] Connection header received
0807 19:47:22.515422 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.515427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.515505 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.515605 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.516605 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.517480 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.519147 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.520057 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.520165 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.523967 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.523973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.523979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.524019 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.528074 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.528940 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.528945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.528948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.529099 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.529524 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.529528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.529531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.530001 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.530136 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.533152 (+ 3016us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.533421 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:47:22.540916 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: Bootstrap starting.
I20260807 19:47:22.533277 15109 master_runner.cc:186] Error getting master registration for 127.0.198.253:42759: OK, instance_id { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" instance_seqno: 1786132041857077 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:22.542285 15109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:22.545109 15338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:22.545362 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.517938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.0.198.253:42759)
0807 19:47:22.518171 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.518179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.518203 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:47:22.518280 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.518288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.518381 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.518508 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.520361 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.522282 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.524412 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.525085 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.525275 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.531877 (+ 6602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.531888 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.531896 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.531955 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.541379 (+ 9424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.542655 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.542662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.542665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.542868 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.543411 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.543417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.543420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.543829 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.544117 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.544732 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.545012 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 19:47:22.545320 15360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:22.545749 15360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:22.546491 15338 log.cc:824] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:22.549984 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: No bootstrap required, opened a new log
I20260807 19:47:22.551889 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:22.559062 15338 raft_consensus.cc:348] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.559731 15338 raft_consensus.cc:374] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:22.559873 15338 raft_consensus.cc:729] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b89c9cfe8db8429db0398c56aa233383, State: Initialized, Role: FOLLOWER
I20260807 19:47:22.560794 15338 consensus_queue.cc:260] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.572065 15369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.572611 15369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.574043 15338 sys_catalog.cc:564] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:22.594072 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.558795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.0.198.254:40209)
0807 19:47:22.558978 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.558983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.568168 (+ 9185us) server_negotiation.cc:407] Connection header received
0807 19:47:22.569774 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.569782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.569874 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.569957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.574453 (+ 4496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.575738 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.578175 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.578923 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.579031 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.584410 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.584422 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.584430 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.584495 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.589068 (+ 4573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.590476 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.590483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.590487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.590666 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.591170 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.591176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.591179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.591979 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.592279 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.593372 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.593669 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:47:22.598213 15258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } attempt: 5
W20260807 19:47:22.618597 15381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 19:47:22.618760 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.568491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.0.198.253:42759)
0807 19:47:22.568693 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.568699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.595324 (+ 26625us) server_negotiation.cc:407] Connection header received
0807 19:47:22.595697 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.595707 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.595834 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.595925 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.603424 (+ 7499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.604478 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.606551 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.607164 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.607285 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.611008 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.611015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.611020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.611063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.615046 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.616119 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.616125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.616128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.616310 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.616809 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.616814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.616817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.617197 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.617454 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.618107 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.618403 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:47:22.619234 15381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:22.623296 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.558723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.0.198.252:39835)
0807 19:47:22.558924 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.558931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.573910 (+ 14979us) server_negotiation.cc:407] Connection header received
0807 19:47:22.574444 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.574454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.574570 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.574710 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.578028 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.579499 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.582496 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.583560 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.583704 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.588967 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.588979 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.588987 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.589056 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.593880 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.619552 (+ 25672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.619558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.619561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.619737 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.620266 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.620271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.620274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.620707 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.621513 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.622592 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.622895 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 19:47:22 dist-test-slave-btw8 krb5kdc[15093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:47:22.609781 15267 master_runner.cc:186] Error getting master registration for 127.0.198.253:42759: OK, instance_id { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" instance_seqno: 1786132041857077 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:22.626160 15267 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:22.634627 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.599040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:39835 (local address 127.0.0.1:52672)
0807 19:47:22.599418 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:22.599441 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:22.599574 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:22.600204 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:22.600212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:22.600724 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:22.601005 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.601024 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.602814 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.602822 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.618590 (+ 15768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:22.618607 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.620601 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.620607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:22.620864 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.621577 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:22.621602 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:22.624484 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:22.629149 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.629157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.629162 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.629812 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.632464 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:22.632471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:22.632475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:22.632680 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:22.633473 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:22.633479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:22.633669 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 19:47:22.633850 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 19:47:22.634215 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260807 19:47:22.634666 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.599099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.0.198.252:39835)
0807 19:47:22.599292 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.599299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.599680 (+ 381us) server_negotiation.cc:407] Connection header received
0807 19:47:22.599775 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.599783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.599886 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.599999 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.601202 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.602613 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.618806 (+ 16193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.619627 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.619788 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.624752 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.624759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.624764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.624809 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.628927 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.632065 (+ 3138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.632072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.632076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.632297 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.632846 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.632851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.632854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.633312 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.633486 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.633997 (+ 511us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.634267 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:47:22.641649 15258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: Bootstrap starting.
I20260807 19:47:22.646317 15258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:22.647662 15347 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:22.647884 15258 log.cc:824] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:22.648038 15347 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.649776 15347 leader_election.cc:288] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ed2cb7921c04ce6a66dce8e86d24058 (127.0.198.253:42759), b89c9cfe8db8429db0398c56aa233383 (127.0.198.252:39835)
I20260807 19:47:22.651039 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b89c9cfe8db8429db0398c56aa233383" is_pre_election: true
I20260807 19:47:22.651759 15311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8bff364c124f3b80aa2e7bdf0d396b in term 0.
I20260807 19:47:22.652498 15127 leader_election.cc:302] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b89c9cfe8db8429db0398c56aa233383, bf8bff364c124f3b80aa2e7bdf0d396b; no voters:
I20260807 19:47:22.657998 15258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: No bootstrap required, opened a new log
I20260807 19:47:22.658288 15347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:22.658495 15347 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:22.658619 15347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:22.662107 15347 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.665271 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b89c9cfe8db8429db0398c56aa233383"
I20260807 19:47:22.665552 15311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:22.669368 15311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8bff364c124f3b80aa2e7bdf0d396b in term 1.
I20260807 19:47:22.670071 15127 leader_election.cc:302] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b89c9cfe8db8429db0398c56aa233383, bf8bff364c124f3b80aa2e7bdf0d396b; no voters:
I20260807 19:47:22.670679 15258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.671242 15258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:22.671489 15258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed2cb7921c04ce6a66dce8e86d24058, State: Initialized, Role: FOLLOWER
I20260807 19:47:22.672420 15258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.675820 15384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:22.676676 15347 leader_election.cc:288] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [CANDIDATE]: Term 1 election: Requested vote from peers 5ed2cb7921c04ce6a66dce8e86d24058 (127.0.198.253:42759), b89c9cfe8db8429db0398c56aa233383 (127.0.198.252:39835)
W20260807 19:47:22.677639 15289 tablet.cc:2369] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:22.679621 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.682313 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.682155 15258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:22.682235 15229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2cb7921c04ce6a66dce8e86d24058"
I20260807 19:47:22.682922 15229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:22.686833 15229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8bff364c124f3b80aa2e7bdf0d396b in term 1.
I20260807 19:47:22.687736 15230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" is_pre_election: true
I20260807 19:47:22.688251 15230 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bf8bff364c124f3b80aa2e7bdf0d396b in term 1. Re-sending same reply.
I20260807 19:47:22.688603 15384 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [term 1 LEADER]: Becoming Leader. State: Replica: bf8bff364c124f3b80aa2e7bdf0d396b, State: Running, Role: LEADER
I20260807 19:47:22.689370 15384 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } }
I20260807 19:47:22.700062 15347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8bff364c124f3b80aa2e7bdf0d396b. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.700459 15347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:22.711603 15390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:22.719985 15390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:22.741655 15311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [term 1 FOLLOWER]: Refusing update from remote peer bf8bff364c124f3b80aa2e7bdf0d396b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:22.743163 15347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:22.754894 15229 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [term 1 FOLLOWER]: Refusing update from remote peer bf8bff364c124f3b80aa2e7bdf0d396b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:22.756260 15347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:22.780880 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8bff364c124f3b80aa2e7bdf0d396b. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.781292 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.783308 15369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8bff364c124f3b80aa2e7bdf0d396b. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.786876 15369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.790382 15394 mvcc.cc:205] Tried to move back new op lower bound from 7315996847054974976 to 7315996846861795328. Current Snapshot: MvccSnapshot[applied={T|T < 7315996847054974976}]
I20260807 19:47:22.809063 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.809406 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 19:47:22.811055 15406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:22.811183 15406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:22.812252 15347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.812559 15347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:22.812867 15347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.813156 15347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:22.819517 15369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8bff364c124f3b80aa2e7bdf0d396b" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 40209 } } peers { permanent_uuid: "5ed2cb7921c04ce6a66dce8e86d24058" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 42759 } } peers { permanent_uuid: "b89c9cfe8db8429db0398c56aa233383" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 39835 } } }
I20260807 19:47:22.819835 15369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:22.820384 15390 catalog_manager.cc:1347] Generated new cluster ID: 6a933757f0a446d8a62654a2914024bb
I20260807 19:47:22.820515 15390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:22.881158 15390 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:22.883140 15390 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:22.905927 15390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf8bff364c124f3b80aa2e7bdf0d396b: Generated new TSK 0
I20260807 19:47:22.907236 15390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:23.622104 15381 catalog_manager.cc:1259] Loaded cluster ID: 6a933757f0a446d8a62654a2914024bb
I20260807 19:47:23.622220 15381 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: loading cluster ID for follower catalog manager: success
I20260807 19:47:23.625037 15381 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: acquiring CA information for follower catalog manager: success
I20260807 19:47:23.626984 15381 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b89c9cfe8db8429db0398c56aa233383: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:47:23.813470 15406 catalog_manager.cc:1259] Loaded cluster ID: 6a933757f0a446d8a62654a2914024bb
I20260807 19:47:23.813603 15406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: loading cluster ID for follower catalog manager: success
I20260807 19:47:23.816589 15406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: acquiring CA information for follower catalog manager: success
I20260807 19:47:23.819108 15406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ed2cb7921c04ce6a66dce8e86d24058: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:47:24.057554 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15109
I20260807 19:47:24.059387 15174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:24.175099 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15109
I20260807 19:47:24.201826 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15188
I20260807 19:47:24.203195 15252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:24.314442 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15188
I20260807 19:47:24.338897 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15267
I20260807 19:47:24.340232 15332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:24.455058 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15267
2026-08-07T19:47:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5154 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:24 dist-test-slave-btw8 krb5kdc[15421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:24 dist-test-slave-btw8 krb5kdc[15421](info): set up 2 sockets
Aug 07 19:47:24 dist-test-slave-btw8 krb5kdc[15421](info): commencing operation
krb5kdc: starting...
W20260807 19:47:25.649258 795 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:47:25 dist-test-slave-btw8 krb5kdc[15421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:47:25.989256 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36147
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:39493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:36147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:26.356877 15435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:26.357224 15435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:26.357383 15435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:26.365088 15435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:26.365231 15435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:26.365309 15435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:26.365403 15435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:26.365465 15435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:26.376627 15435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:39493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:36147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:26.378181 15435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:26.379777 15435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:26.392639 15440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:26.392812 15443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:26.392606 15441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:26.393919 15435 server_base.cc:1034] running on GCE node
I20260807 19:47:26.394908 15435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:26.396634 15435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:26.397914 15435 hybrid_clock.cc:648] HybridClock initialized: now 1786132046397859 us; error 61 us; skew 500 ppm
I20260807 19:47:26.398312 15435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:26 dist-test-slave-btw8 krb5kdc[15421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132046, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:26.402086 15435 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:26.403995 15435 webserver.cc:460] Webserver started at http://127.0.198.254:46841/ using document root <none> and password file <none>
I20260807 19:47:26.404892 15435 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:26.405073 15435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:26.405464 15435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:26.409195 15435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "5f80e31581744b76b0b11d833f92d843"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-btw8"
server_key: "2f152abef6e12fbc0d9d39ac942c359a"
server_key_iv: "30ffd5e0e7b8c89edd5c81dc72ab41ce"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.410219 15435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "5f80e31581744b76b0b11d833f92d843"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-btw8"
server_key: "2f152abef6e12fbc0d9d39ac942c359a"
server_key_iv: "30ffd5e0e7b8c89edd5c81dc72ab41ce"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.416852 15435 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:47:26.421590 15452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:26.423604 15435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:26.423944 15435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "5f80e31581744b76b0b11d833f92d843"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-btw8"
server_key: "2f152abef6e12fbc0d9d39ac942c359a"
server_key_iv: "30ffd5e0e7b8c89edd5c81dc72ab41ce"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.424244 15435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:26.448315 15435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:26.453449 15435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:26.453774 15435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:26.476610 15435 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:36147
I20260807 19:47:26.476658 15508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:36147 every 8 connection(s)
I20260807 19:47:26.478304 15435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:26.482925 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:26.484476 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15435
I20260807 19:47:26.484812 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:47:26.485657 795 external_mini_cluster.cc:1428] Setting key 053f0094dccb059627b71386be061fb0
I20260807 19:47:26.500092 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843: Bootstrap starting.
Aug 07 19:47:26 dist-test-slave-btw8 krb5kdc[15421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:26.504683 15509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:26.506389 15509 log.cc:824] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:26.510950 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843: No bootstrap required, opened a new log
I20260807 19:47:26.516700 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:26.487946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.0.198.254:36147)
0807 19:47:26.488910 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:26.488929 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:26.488986 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:47:26.490931 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:26.490971 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:26.491659 (+ 688us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:26.492164 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:26.493714 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:26.495896 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:26.497688 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:26.498982 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:26.499205 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:26.505092 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:26.505111 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:26.505143 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:26.505241 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:26.510291 (+ 5050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:26.511556 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:26.511569 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:26.511582 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:26.511897 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:26.512436 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:26.512442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:26.512445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:26.513354 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:26.513754 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:26.514333 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 19:47:26.514922 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":420,"threads_started":1}
I20260807 19:47:26.522490 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:26.524575 15509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } }
I20260807 19:47:26.525137 15509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:26.525336 15509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f80e31581744b76b0b11d833f92d843, State: Initialized, Role: FOLLOWER
I20260807 19:47:26.526137 15509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } }
I20260807 19:47:26.526544 15509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:26.526778 15509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:26.527007 15509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:26.530169 15509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } }
I20260807 19:47:26.530900 15509 leader_election.cc:302] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f80e31581744b76b0b11d833f92d843; no voters:
I20260807 19:47:26.531750 15509 leader_election.cc:288] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:26.531957 15516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:26.532776 15516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [term 1 LEADER]: Becoming Leader. State: Replica: 5f80e31581744b76b0b11d833f92d843, State: Running, Role: LEADER
I20260807 19:47:26.533502 15516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } }
I20260807 19:47:26.534186 15509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:26.537750 15517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f80e31581744b76b0b11d833f92d843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } } }
I20260807 19:47:26.537395 15518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f80e31581744b76b0b11d833f92d843. Latest consensus state: current_term: 1 leader_uuid: "5f80e31581744b76b0b11d833f92d843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f80e31581744b76b0b11d833f92d843" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36147 } } }
I20260807 19:47:26.538157 15517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:26.538169 15518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:26.540738 15522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:26.546495 15522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:26.558369 15522 catalog_manager.cc:1347] Generated new cluster ID: d153594e790545489da8779126db2325
I20260807 19:47:26.558547 15522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:26.595928 15522 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:26.597613 15522 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:26.608493 15522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f80e31581744b76b0b11d833f92d843: Generated new TSK 0
I20260807 19:47:26.609630 15522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:26.637799 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15435
I20260807 19:47:26.639221 15504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:26.743973 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15435
2026-08-07T19:47:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2284 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:26 dist-test-slave-btw8 krb5kdc[15541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:26 dist-test-slave-btw8 krb5kdc[15541](info): set up 2 sockets
Aug 07 19:47:26 dist-test-slave-btw8 krb5kdc[15541](info): commencing operation
krb5kdc: starting...
W20260807 19:47:27.966861 795 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:47:28 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:47:28.316828 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:28.698815 15554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:28.699239 15554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:28.699479 15554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:28.707237 15554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:28.707491 15554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:28.707600 15554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:28.707715 15554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:28.707804 15554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:28.718343 15554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:44765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:28.719965 15554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:28.721374 15554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:28.735066 15563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:28.735117 15561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:28.735196 15560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:28.736229 15554 server_base.cc:1034] running on GCE node
I20260807 19:47:28.737107 15554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:28.738854 15554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:28.740095 15554 hybrid_clock.cc:648] HybridClock initialized: now 1786132048740024 us; error 71 us; skew 500 ppm
I20260807 19:47:28.740553 15554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:28 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:28.744655 15554 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:28.747009 15554 webserver.cc:460] Webserver started at http://127.0.198.254:34815/ using document root <none> and password file <none>
I20260807 19:47:28.747997 15554 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:28.748179 15554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:28.748572 15554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:28.752435 15554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "16852ead1388436db85f4781562507d3"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-btw8"
server_key: "53e97249c35828fb79595ca9685f2507"
server_key_iv: "faef0a84eb57ce0f5b51c50a9d9e4cec"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.753499 15554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "16852ead1388436db85f4781562507d3"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-btw8"
server_key: "53e97249c35828fb79595ca9685f2507"
server_key_iv: "faef0a84eb57ce0f5b51c50a9d9e4cec"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.760002 15554 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:47:28.764470 15570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:28.766263 15554 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:47:28.766518 15554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "16852ead1388436db85f4781562507d3"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-btw8"
server_key: "53e97249c35828fb79595ca9685f2507"
server_key_iv: "faef0a84eb57ce0f5b51c50a9d9e4cec"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.766791 15554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:28.781474 15554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:28.786738 15554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:28.787115 15554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:28.810653 15554 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:44765
I20260807 19:47:28.810786 15625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:44765 every 8 connection(s)
I20260807 19:47:28.812423 15554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:28.817008 15628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:28.822636 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15554
I20260807 19:47:28.822945 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:47:28.823747 795 external_mini_cluster.cc:1428] Setting key 79c35863e97202d15373768342750f2d
I20260807 19:47:28.831905 15628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap starting.
I20260807 19:47:28.836110 15628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:28.837432 15628 log.cc:824] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:28.841722 15628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: No bootstrap required, opened a new log
Aug 07 19:47:28 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:28.852628 15628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:28.853245 15628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:28.853375 15628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16852ead1388436db85f4781562507d3, State: Initialized, Role: FOLLOWER
I20260807 19:47:28.854163 15628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:28.854496 15628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:28.854662 15628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:28.854890 15628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:28.858512 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:28.827558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.0.198.254:44765)
0807 19:47:28.828846 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:28.828866 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:28.828934 (+ 68us) server_negotiation.cc:407] Connection header received
0807 19:47:28.831581 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:28.831617 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:28.832221 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:28.832733 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:28.834309 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.836396 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.838431 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.839302 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.839631 (+ 329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:28.845676 (+ 6045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:28.845690 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:28.845715 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:28.845778 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:28.850982 (+ 5204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.852606 (+ 1624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.852620 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.852628 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.852891 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.853889 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.853894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.853898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.854803 (+ 905us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:28.855185 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:28.856329 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0807 19:47:28.856911 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":599,"threads_started":1}
I20260807 19:47:28.858543 15628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:28.859220 15628 leader_election.cc:302] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16852ead1388436db85f4781562507d3; no voters:
I20260807 19:47:28.859997 15628 leader_election.cc:288] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:28.860138 15635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:28.862185 15628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:28.862741 15635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 LEADER]: Becoming Leader. State: Replica: 16852ead1388436db85f4781562507d3, State: Running, Role: LEADER
I20260807 19:47:28.863433 15635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:28.867022 15637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16852ead1388436db85f4781562507d3. Latest consensus state: current_term: 1 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:28.867283 15636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:28.867583 15637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:28.867699 15636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:28.868737 15645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:28.875613 15645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:28.886655 15645 catalog_manager.cc:1347] Generated new cluster ID: 10a14db6b7454a4e8b849127b7e1f13e
I20260807 19:47:28.886821 15645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:28.911600 15645 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:28.913688 15645 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:28.930487 15645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Generated new TSK 0
I20260807 19:47:28.931746 15645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:47:29.024923 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:29.431656 15656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:29.432008 15656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:29.432175 15656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:29.440513 15656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:29.440685 15656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:29.440917 15656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:29.452848 15656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:29.454372 15656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:29.455868 15656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:29.470731 15662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:29.470788 15661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:29.471187 15664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:29.472497 15656 server_base.cc:1034] running on GCE node
I20260807 19:47:29.473229 15656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:29.474380 15656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:29.475677 15656 hybrid_clock.cc:648] HybridClock initialized: now 1786132049475634 us; error 40 us; skew 500 ppm
I20260807 19:47:29.476118 15656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:29 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:29.479995 15656 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:29.482075 15656 webserver.cc:460] Webserver started at http://127.0.198.193:40703/ using document root <none> and password file <none>
I20260807 19:47:29.483155 15656 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:29.483433 15656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:29.483880 15656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:29.488132 15656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "6678989673b945e89b0f188882e2bdbc"
format_stamp: "Formatted at 2026-08-07 19:47:29 on dist-test-slave-btw8"
server_key: "deaea9e29f73c69fe16a20e7d7e729b3"
server_key_iv: "a6c868d7da325c1c5aad4824b154ebc2"
server_key_version: "encryptionkey@0"
I20260807 19:47:29.489166 15656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "6678989673b945e89b0f188882e2bdbc"
format_stamp: "Formatted at 2026-08-07 19:47:29 on dist-test-slave-btw8"
server_key: "deaea9e29f73c69fe16a20e7d7e729b3"
server_key_iv: "a6c868d7da325c1c5aad4824b154ebc2"
server_key_version: "encryptionkey@0"
I20260807 19:47:29.496412 15656 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 19:47:29.500429 15672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:29.501977 15656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:47:29.502204 15656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "6678989673b945e89b0f188882e2bdbc"
format_stamp: "Formatted at 2026-08-07 19:47:29 on dist-test-slave-btw8"
server_key: "deaea9e29f73c69fe16a20e7d7e729b3"
server_key_iv: "a6c868d7da325c1c5aad4824b154ebc2"
server_key_version: "encryptionkey@0"
I20260807 19:47:29.502449 15656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:29.519696 15656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:29.525233 15656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:29.525619 15656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:29.526832 15656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:29.528616 15656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:29.528779 15656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:47:29.528934 15656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:29.529004 15656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:29.566623 15656 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:36075
I20260807 19:47:29.566668 15795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:36075 every 8 connection(s)
I20260807 19:47:29.568166 15656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:29.569923 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15656
I20260807 19:47:29.570159 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:47:29.570943 795 external_mini_cluster.cc:1428] Setting key f48483c8b559ecb5cb400acdfdcd0399
Aug 07 19:47:29 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:29.598731 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:29.571140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:37509 (local address 127.0.198.254:44765)
0807 19:47:29.571794 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:29.571802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:29.573041 (+ 1239us) server_negotiation.cc:407] Connection header received
0807 19:47:29.575344 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:29.575379 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:29.575491 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:29.575650 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:29.577587 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:29.578455 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:29.580582 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:29.581064 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:29.581232 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:29.591306 (+ 10074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:29.591313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:29.591316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:29.591426 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:29.594842 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:29.596005 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:29.596013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:29.596022 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:29.596171 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:29.596706 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:29.596710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:29.596713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:29.597037 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:29.597220 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:29.598040 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 19:47:29.598319 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":339,"threads_started":1}
I20260807 19:47:29.599925 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:29.571922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:37509)
0807 19:47:29.572836 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:29.572895 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:29.575001 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:29.575964 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:29.575974 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:29.576604 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:29.577298 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:29.577320 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:29.578693 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:29.578698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:29.580352 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:29.580370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:29.581394 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:29.581404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:29.581685 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:29.587764 (+ 6079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:29.587806 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:29.590904 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:29.595122 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:29.595132 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:29.595150 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:29.595826 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:29.596330 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:29.596335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:29.596337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:29.596562 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:29.597192 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:29.597202 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:29.597623 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 19:47:29.597928 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 19:47:29.598569 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":471,"threads_started":1}
I20260807 19:47:29.601737 15796 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:29.602121 15796 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:29.603190 15796 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
I20260807 19:47:29.605851 15589 ts_manager.cc:194] Registered new tserver with Master: 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
I20260807 19:47:29.607833 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:37509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:47:29.647886 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:30.026850 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:30.027174 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:30.027288 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:30.034905 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:30.035045 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:30.035164 15802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:30.045598 15802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:30.046962 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:30.048410 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:30.063264 15807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.064505 15810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.065021 15808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:30.065162 15802 server_base.cc:1034] running on GCE node
I20260807 19:47:30.065915 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:30.067130 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:30.068406 15802 hybrid_clock.cc:648] HybridClock initialized: now 1786132050068325 us; error 97 us; skew 500 ppm
I20260807 19:47:30.068833 15802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:30 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:30.072681 15802 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:47:30.074445 15802 webserver.cc:460] Webserver started at http://127.0.198.194:33463/ using document root <none> and password file <none>
I20260807 19:47:30.075238 15802 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:30.075412 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:30.075781 15802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:30.078953 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "a04e2f276f7d43ff82ef124ed59537f1"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "672c04cdfbb2896e08cbceb49df5979f"
server_key_iv: "33a819e7d995badb40f4072ea8d8676c"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.079869 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "a04e2f276f7d43ff82ef124ed59537f1"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "672c04cdfbb2896e08cbceb49df5979f"
server_key_iv: "33a819e7d995badb40f4072ea8d8676c"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.086006 15802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 19:47:30.091288 15817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.093330 15802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:47:30.093592 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "a04e2f276f7d43ff82ef124ed59537f1"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "672c04cdfbb2896e08cbceb49df5979f"
server_key_iv: "33a819e7d995badb40f4072ea8d8676c"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.093850 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:30.113418 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:30.118462 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:30.118824 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:30.120225 15802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:30.121861 15802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:30.121978 15802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.122165 15802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:30.122248 15802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.161321 15802 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:43533
I20260807 19:47:30.161353 15941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:43533 every 8 connection(s)
I20260807 19:47:30.162999 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:30.165886 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15802
I20260807 19:47:30.166138 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:47:30.166803 795 external_mini_cluster.cc:1428] Setting key 4d062ee7d198a34422e1e49eb7dfbdb5
Aug 07 19:47:30 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:30.189754 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.166363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:44257 (local address 127.0.198.254:44765)
0807 19:47:30.166958 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.166973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.168401 (+ 1428us) server_negotiation.cc:407] Connection header received
0807 19:47:30.170793 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.170802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.170914 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.171041 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:30.172945 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.174127 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.176593 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.177420 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.177586 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.182268 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:30.182278 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:30.182286 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:30.182342 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:30.185445 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.186785 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.186797 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.186803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.186999 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.187668 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.187674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.187676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.188001 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:30.188167 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.189219 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.189447 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":283,"threads_started":1}
I20260807 19:47:30.191413 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.167052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:44257)
0807 19:47:30.168208 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.168267 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.170454 (+ 2187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.171384 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.171401 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.172068 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:30.172743 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.172764 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.174355 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.174361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.176367 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.176385 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.177788 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.177795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.178065 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.178968 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:30.179001 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:30.181915 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:30.185648 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:30.185662 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:30.185687 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:30.186510 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:30.187143 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:30.187151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:30.187157 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:30.187506 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:30.188161 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:30.188173 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:30.188727 (+ 554us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.189071 (+ 344us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.189727 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":430,"threads_started":1}
I20260807 19:47:30.193323 15942 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:30.193732 15942 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:30.194591 15942 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
I20260807 19:47:30.196651 15589 ts_manager.cc:194] Registered new tserver with Master: a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533)
I20260807 19:47:30.198060 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:44257
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:47:30.245193 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:30.611515 15796 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
W20260807 19:47:30.623099 15949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:30.623517 15949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:30.623695 15949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:30.631551 15949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:30.631680 15949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:30.631876 15949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:30.643090 15949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:30.644629 15949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:30.646092 15949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:30.661453 15954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.661556 15955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.661566 15957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:30.662998 15949 server_base.cc:1034] running on GCE node
I20260807 19:47:30.663908 15949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:30.665133 15949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:30.666435 15949 hybrid_clock.cc:648] HybridClock initialized: now 1786132050666371 us; error 48 us; skew 500 ppm
I20260807 19:47:30.666944 15949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:30 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:30.670959 15949 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:30.672904 15949 webserver.cc:460] Webserver started at http://127.0.198.195:45475/ using document root <none> and password file <none>
I20260807 19:47:30.673674 15949 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:30.673801 15949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:30.674182 15949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:30.677692 15949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "92655035e3d84ae49d84f6c0d537ca4c"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "0a1af2bb07cb0ccd3eda0140397012ad"
server_key_iv: "ed72ac816c6371a749bbe65d46665968"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.678640 15949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "92655035e3d84ae49d84f6c0d537ca4c"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "0a1af2bb07cb0ccd3eda0140397012ad"
server_key_iv: "ed72ac816c6371a749bbe65d46665968"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.685007 15949 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:47:30.689316 15964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.691201 15949 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 19:47:30.691514 15949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "92655035e3d84ae49d84f6c0d537ca4c"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "0a1af2bb07cb0ccd3eda0140397012ad"
server_key_iv: "ed72ac816c6371a749bbe65d46665968"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.691807 15949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:30.723623 15949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:30.728804 15949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:30.729166 15949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:30.730316 15949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:30.731988 15949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:30.732116 15949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.732287 15949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:30.732371 15949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.772825 15949 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:46187
I20260807 19:47:30.772881 16088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:46187 every 8 connection(s)
I20260807 19:47:30.774228 15949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:30.782122 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 15949
I20260807 19:47:30.782328 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:47:30.782917 795 external_mini_cluster.cc:1428] Setting key 2030d8912de126e714f02b6a135a3887
Aug 07 19:47:30 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:30.799676 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.777415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:45643 (local address 127.0.198.254:44765)
0807 19:47:30.777980 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.777985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.778942 (+ 957us) server_negotiation.cc:407] Connection header received
0807 19:47:30.780975 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.780981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.781113 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.781241 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:30.783616 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.784786 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.787071 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.787576 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.787865 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.792288 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:30.792294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:30.792298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:30.792338 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:30.795630 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.796612 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.796616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.796619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.796783 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.797317 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.797321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.797323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.797705 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:30.797967 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.798902 (+ 935us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.799196 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":309,"threads_started":1}
I20260807 19:47:30.800745 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.777920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:45643)
0807 19:47:30.778744 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.778803 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.780665 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.781510 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.781518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.782400 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:30.783304 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.783336 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.784995 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.785001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.786915 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.786930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.787905 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.787912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.788166 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.789049 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:30.789078 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:30.791966 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:30.795837 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:30.795846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:30.795861 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:30.796438 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:30.796906 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:30.796909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:30.796912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:30.797169 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:30.797954 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:30.797967 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:30.798430 (+ 463us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.798813 (+ 383us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.799445 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":318,"threads_started":1}
I20260807 19:47:30.802125 16089 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:30.802413 16089 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:30.803083 16089 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
I20260807 19:47:30.804557 15589 ts_manager.cc:194] Registered new tserver with Master: 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:30.805415 15589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:45643
I20260807 19:47:30.812211 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:30.832993 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.818328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.0.198.254:44765)
0807 19:47:30.818542 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.818556 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.818982 (+ 426us) server_negotiation.cc:407] Connection header received
0807 19:47:30.819172 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.819179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.819284 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.819444 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:30.821084 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.821871 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.823420 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.823929 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.824043 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.827104 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:30.827110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:30.827113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:30.827157 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:30.830075 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.830958 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.830963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.830965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.831091 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:30.831542 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:30.831546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:30.831548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:30.831911 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:30.832096 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.832503 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.832708 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:47:30.838446 15589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:30.845791 15589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:30.876399 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.860084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43533 (local address 127.0.0.1:56590)
0807 19:47:30.861509 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.861566 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.861817 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.863929 (+ 2112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.863936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.863989 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:30.864466 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.864477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.867858 (+ 3381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.867865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.871602 (+ 3737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.871628 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.874634 (+ 3006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.874647 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.875057 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.875152 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.875323 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.875699 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"threads_started":1}
I20260807 19:47:30.879019 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.860774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.0.198.194:43533)
0807 19:47:30.863136 (+ 2362us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.863159 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.863192 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:47:30.863303 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.863312 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.863638 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.864715 (+ 1077us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:30.864997 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.867653 (+ 2656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.871940 (+ 4287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.874275 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.878028 (+ 3753us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.878311 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.878404 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.878620 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2189,"thread_start_us":2081,"threads_started":1}
I20260807 19:47:30.885762 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.860763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.0.198.193:36075)
0807 19:47:30.862662 (+ 1899us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.862670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.873593 (+ 10923us) server_negotiation.cc:407] Connection header received
0807 19:47:30.874296 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.874312 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.874739 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.875030 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:30.876846 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.879676 (+ 2830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.882303 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.883954 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.884117 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.884362 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.885199 (+ 837us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.885409 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1705,"thread_start_us":1505,"threads_started":1}
I20260807 19:47:30.885118 15873 tablet_service.cc:1467] Processing CreateTablet for tablet 26e19898905344a19c81415755e3716c (DEFAULT_TABLE table=test-table [id=fa94d26708e9409c97ab0b1210919338]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:30.886669 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.860344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59590)
0807 19:47:30.873385 (+ 13041us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.873416 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.873976 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.875601 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.875606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.875649 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:30.876321 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.876344 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.879888 (+ 3544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.879898 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.882086 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.882096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.884558 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.884565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.884897 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.884935 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.886012 (+ 1077us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.886249 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12744,"mutex_wait_us":70,"thread_start_us":593,"threads_started":1}
I20260807 19:47:30.887189 15873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e19898905344a19c81415755e3716c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:30.888805 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.865013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.0.198.195:46187)
0807 19:47:30.880925 (+ 15912us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.880935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.880973 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:47:30.881082 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.881090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.881407 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.881651 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:30.882538 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.883764 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.886068 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.887316 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.887495 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.887748 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.888009 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.888370 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15704,"thread_start_us":15509,"threads_started":1}
I20260807 19:47:30.889601 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.864178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46187 (local address 127.0.0.1:45364)
0807 19:47:30.876998 (+ 12820us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.877030 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.877309 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.881675 (+ 4366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.881682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.881726 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:30.882079 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.882096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.883983 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.883989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.885805 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.885821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.887509 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.887515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.887721 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.887760 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.887862 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.888154 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12579,"mutex_wait_us":38,"thread_start_us":9423,"threads_started":1}
I20260807 19:47:30.891539 15724 tablet_service.cc:1467] Processing CreateTablet for tablet 26e19898905344a19c81415755e3716c (DEFAULT_TABLE table=test-table [id=fa94d26708e9409c97ab0b1210919338]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:30.894240 15724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e19898905344a19c81415755e3716c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:30.894315 16018 tablet_service.cc:1467] Processing CreateTablet for tablet 26e19898905344a19c81415755e3716c (DEFAULT_TABLE table=test-table [id=fa94d26708e9409c97ab0b1210919338]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:30.895998 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e19898905344a19c81415755e3716c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:30.902560 16108 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Bootstrap starting.
I20260807 19:47:30.906219 16108 tablet_bootstrap.cc:654] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:30.907718 16108 log.cc:824] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:30.912879 16110 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Bootstrap starting.
I20260807 19:47:30.914765 16111 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Bootstrap starting.
I20260807 19:47:30.915115 16108 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: No bootstrap required, opened a new log
I20260807 19:47:30.915567 16108 ts_tablet_manager.cc:1397] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.007s
I20260807 19:47:30.917644 16110 tablet_bootstrap.cc:654] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:30.918111 16111 tablet_bootstrap.cc:654] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:30.919086 16110 log.cc:824] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:30.919202 16111 log.cc:824] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:30.922974 16110 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: No bootstrap required, opened a new log
I20260807 19:47:30.922967 16111 tablet_bootstrap.cc:492] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: No bootstrap required, opened a new log
I20260807 19:47:30.923326 16110 ts_tablet_manager.cc:1397] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 19:47:30.923326 16111 ts_tablet_manager.cc:1397] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 19:47:30.925774 16108 raft_consensus.cc:348] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.926352 16108 raft_consensus.cc:374] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:30.926460 16108 raft_consensus.cc:729] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04e2f276f7d43ff82ef124ed59537f1, State: Initialized, Role: FOLLOWER
I20260807 19:47:30.927209 16108 consensus_queue.cc:260] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.928723 15942 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
I20260807 19:47:30.929539 16108 ts_tablet_manager.cc:1428] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260807 19:47:30.935611 16114 raft_consensus.cc:482] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:30.935371 16110 raft_consensus.cc:348] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.935369 16111 raft_consensus.cc:348] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.935909 16110 raft_consensus.cc:374] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:30.935909 16111 raft_consensus.cc:374] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:30.936024 16111 raft_consensus.cc:729] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6678989673b945e89b0f188882e2bdbc, State: Initialized, Role: FOLLOWER
I20260807 19:47:30.936024 16110 raft_consensus.cc:729] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92655035e3d84ae49d84f6c0d537ca4c, State: Initialized, Role: FOLLOWER
I20260807 19:47:30.935950 16114 raft_consensus.cc:504] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.936761 16110 consensus_queue.cc:260] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.936775 16111 consensus_queue.cc:260] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.938661 16089 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
I20260807 19:47:30.939065 16111 ts_tablet_manager.cc:1428] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260807 19:47:30.939194 16110 ts_tablet_manager.cc:1428] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Time spent starting tablet: real 0.016s user 0.013s sys 0.002s
I20260807 19:47:30.939659 16114 leader_election.cc:288] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:30.948617 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.939013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:60043 (local address 127.0.198.193:36075)
0807 19:47:30.939725 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.939735 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.940720 (+ 985us) server_negotiation.cc:407] Connection header received
0807 19:47:30.941045 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.941051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.941127 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.941196 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:30.942080 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.943160 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.946065 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.947434 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.947566 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.947641 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.948244 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.948404 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488}
I20260807 19:47:30.948722 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.938465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.198.194:60043)
0807 19:47:30.940579 (+ 2114us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.940611 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.940855 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.941437 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.941442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.941471 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:30.941913 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.941930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.943386 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.943394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.945862 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.945877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.947884 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.947891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.948055 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.948084 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.948182 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.948297 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":1747,"threads_started":1}
I20260807 19:47:30.950719 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.941586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:57959 (local address 127.0.198.195:46187)
0807 19:47:30.941834 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.941843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.943476 (+ 1633us) server_negotiation.cc:407] Connection header received
0807 19:47:30.943780 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.943786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.943874 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.943937 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:30.944899 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.946010 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.948474 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.949688 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.949829 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.949886 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.950294 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.950434 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:47:30.950773 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.940762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46187 (local address 127.0.198.194:57959)
0807 19:47:30.943378 (+ 2616us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:30.943397 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:30.943604 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:30.944174 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:30.944180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:30.944213 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:30.944676 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.944695 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.946192 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.946198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.948291 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:30.948307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.949906 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.949911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:30.950115 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.950141 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:47:30.950290 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:47:30.950436 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2438,"thread_start_us":2247,"threads_started":1}
I20260807 19:47:30.950749 15745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e19898905344a19c81415755e3716c" candidate_uuid: "a04e2f276f7d43ff82ef124ed59537f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6678989673b945e89b0f188882e2bdbc" is_pre_election: true
I20260807 19:47:30.951557 15745 raft_consensus.cc:2454] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a04e2f276f7d43ff82ef124ed59537f1 in term 0.
I20260807 19:47:30.952175 16039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e19898905344a19c81415755e3716c" candidate_uuid: "a04e2f276f7d43ff82ef124ed59537f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92655035e3d84ae49d84f6c0d537ca4c" is_pre_election: true
I20260807 19:47:30.952466 15822 leader_election.cc:302] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc, a04e2f276f7d43ff82ef124ed59537f1; no voters:
I20260807 19:47:30.952723 16039 raft_consensus.cc:2454] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a04e2f276f7d43ff82ef124ed59537f1 in term 0.
I20260807 19:47:30.952915 16114 raft_consensus.cc:2781] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:30.953102 16114 raft_consensus.cc:482] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:30.953212 16114 raft_consensus.cc:3037] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:30.955672 16114 raft_consensus.cc:504] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.956313 16114 leader_election.cc:288] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [CANDIDATE]: Term 1 election: Requested vote from peers 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:30.957042 15745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e19898905344a19c81415755e3716c" candidate_uuid: "a04e2f276f7d43ff82ef124ed59537f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6678989673b945e89b0f188882e2bdbc"
I20260807 19:47:30.957147 16039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e19898905344a19c81415755e3716c" candidate_uuid: "a04e2f276f7d43ff82ef124ed59537f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92655035e3d84ae49d84f6c0d537ca4c"
I20260807 19:47:30.957307 15745 raft_consensus.cc:3037] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:30.957434 16039 raft_consensus.cc:3037] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:30.959877 15745 raft_consensus.cc:2454] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a04e2f276f7d43ff82ef124ed59537f1 in term 1.
I20260807 19:47:30.960161 16039 raft_consensus.cc:2454] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a04e2f276f7d43ff82ef124ed59537f1 in term 1.
I20260807 19:47:30.960474 15822 leader_election.cc:302] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc, a04e2f276f7d43ff82ef124ed59537f1; no voters:
I20260807 19:47:30.960858 16114 raft_consensus.cc:2781] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:30.961385 16114 raft_consensus.cc:686] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 1 LEADER]: Becoming Leader. State: Replica: a04e2f276f7d43ff82ef124ed59537f1, State: Running, Role: LEADER
I20260807 19:47:30.962069 16114 consensus_queue.cc:237] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:30.968206 15589 catalog_manager.cc:5495] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "a04e2f276f7d43ff82ef124ed59537f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:31.005616 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.999436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.0.198.194:43533)
0807 19:47:30.999751 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.999761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.999784 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:47:30.999895 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.999899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.999992 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:31.000204 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:31.000639 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.001420 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.002981 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.003528 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.003686 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:31.004337 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:31.004895 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:31.005074 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:31.005184 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 19:47:31.005280 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:47:31.015712 16039 raft_consensus.cc:1261] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Refusing update from remote peer a04e2f276f7d43ff82ef124ed59537f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:31.015743 15745 raft_consensus.cc:1261] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Refusing update from remote peer a04e2f276f7d43ff82ef124ed59537f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:31.017132 16119 consensus_queue.cc:1035] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:31.017632 16114 consensus_queue.cc:1035] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:47:31.026162 16090 tablet.cc:2369] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:31.032343 16122 mvcc.cc:205] Tried to move back new op lower bound from 7315996880947326976 to 7315996880745885696. Current Snapshot: MvccSnapshot[applied={T|T < 7315996880947326976}]
I20260807 19:47:31.048834 15587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:31.049239 15587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:31.054226 15587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Sending DeleteTablet for 3 replicas of tablet 26e19898905344a19c81415755e3716c
I20260807 19:47:31.055955 15873 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e19898905344a19c81415755e3716c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:31 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56590
I20260807 19:47:31.055943 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e19898905344a19c81415755e3716c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:31 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45364
I20260807 19:47:31.055938 15724 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e19898905344a19c81415755e3716c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:31 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:59590
I20260807 19:47:31.057578 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15656
I20260807 19:47:31.074537 16133 tablet_replica.cc:330] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: stopping tablet replica
I20260807 19:47:31.075291 16135 tablet_replica.cc:330] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: stopping tablet replica
I20260807 19:47:31.076808 16135 raft_consensus.cc:2229] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:31.077566 16135 raft_consensus.cc:2258] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:31.077502 16133 raft_consensus.cc:2229] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:47:31.079016 16133 raft_consensus.cc:2258] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:31.079852 16132 tablet_replica.cc:330] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: stopping tablet replica
I20260807 19:47:31.080605 16132 raft_consensus.cc:2229] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:31.081251 16135 ts_tablet_manager.cc:1905] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:31.081403 16132 raft_consensus.cc:2258] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 19:47:31 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:47:31.084539 16132 ts_tablet_manager.cc:1905] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:31.089689 16133 ts_tablet_manager.cc:1905] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:31.092355 16135 ts_tablet_manager.cc:1918] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:31.092351 16132 ts_tablet_manager.cc:1918] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:31.092633 16132 log.cc:1196] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/26e19898905344a19c81415755e3716c
I20260807 19:47:31.093327 16132 ts_tablet_manager.cc:1939] T 26e19898905344a19c81415755e3716c P 92655035e3d84ae49d84f6c0d537ca4c: Deleting consensus metadata
I20260807 19:47:31.095584 15575 catalog_manager.cc:4841] TS 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): tablet 26e19898905344a19c81415755e3716c (table test-table [id=fa94d26708e9409c97ab0b1210919338]) successfully deleted
I20260807 19:47:31.095209 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.059277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59596 (local address 127.0.198.193:36075)
0807 19:47:31.059570 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:31.059582 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:31.065287 (+ 5705us) server_negotiation.cc:407] Connection header received
0807 19:47:31.066692 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:31.066703 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:31.066826 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:31.066944 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:31.072758 (+ 5814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.074424 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.077195 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.078010 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.078180 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:31.086049 (+ 7869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:31.086070 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:31.086091 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:31.086161 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:31.090897 (+ 4736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.092165 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.092179 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.092183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.092406 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.093017 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.093022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.093026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.093543 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:31.093861 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:31.094518 (+ 657us) server_negotiation.cc:299] Negotiation successful
0807 19:47:31.094801 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 19:47:31.096715 16135 log.cc:1196] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/26e19898905344a19c81415755e3716c
I20260807 19:47:31.097028 15790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:31.102721 16133 ts_tablet_manager.cc:1918] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:31.102952 16133 log.cc:1196] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/26e19898905344a19c81415755e3716c
I20260807 19:47:31.103565 16133 ts_tablet_manager.cc:1939] T 26e19898905344a19c81415755e3716c P a04e2f276f7d43ff82ef124ed59537f1: Deleting consensus metadata
I20260807 19:47:31.105777 15571 catalog_manager.cc:4841] TS a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): tablet 26e19898905344a19c81415755e3716c (table test-table [id=fa94d26708e9409c97ab0b1210919338]) successfully deleted
I20260807 19:47:31.205780 16135 ts_tablet_manager.cc:1939] T 26e19898905344a19c81415755e3716c P 6678989673b945e89b0f188882e2bdbc: Deleting consensus metadata
I20260807 19:47:31.206620 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15656
I20260807 19:47:31.229099 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15802
W20260807 19:47:31.229074 15575 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:36075 (error 108)
I20260807 19:47:31.231596 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.231025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59610)
0807 19:47:31.231244 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.231400 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 19:47:31.231864 15575 catalog_manager.cc:4584] TS 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26e19898905344a19c81415755e3716c: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Aug 07 19:47:31 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:47:31.251700 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.230266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.0.198.194:43533)
0807 19:47:31.230461 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:31.230467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:31.230486 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:31.230555 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:31.230559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:31.230649 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:31.230744 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:31.232881 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.233711 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.235742 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.236305 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.236434 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:31.242499 (+ 6065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:31.242525 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:31.242548 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:31.242610 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:31.247843 (+ 5233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.248812 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.248832 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.248836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.249075 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.249585 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.249591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.249595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.250136 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:31.250548 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:31.250865 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:47:31.251144 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:47:31.252699 15937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:31.285934 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.285322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59612)
0807 19:47:31.285620 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.285741 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 19:47:31.287442 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.286895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59626)
0807 19:47:31.287089 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.287195 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:47:31.348306 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.347734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59638)
0807 19:47:31.347983 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.348109 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:47:31.349736 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.349227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59640)
0807 19:47:31.349409 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.349508 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 19:47:31.378051 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15802
I20260807 19:47:31.402410 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15949
Aug 07 19:47:31 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:47:31.421005 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.403429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.0.198.195:46187)
0807 19:47:31.403629 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:31.403637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:31.403682 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:47:31.403924 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:31.403934 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:31.404002 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:31.404113 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:31.405583 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.406414 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.407939 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:31.408366 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:31.408516 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:31.413542 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:31.413560 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:31.413578 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:31.413624 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:31.417197 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.418178 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.418187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.418190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.418353 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:31.418824 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:31.418829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:31.418831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:31.419289 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:31.419727 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:31.420269 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 19:47:31.420560 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:47:31.421768 16082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:31.430294 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.429702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59642)
0807 19:47:31.429993 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.430109 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 19:47:31.431699 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.431150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59656)
0807 19:47:31.431449 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.431546 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 19:47:31.517067 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15949
I20260807 19:47:31.537400 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 15554
I20260807 19:47:31.538865 15621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:31.655155 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 15554
I20260807 19:47:31.655277 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:31.654798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59664)
0807 19:47:31.655018 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:31.655114 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:47:31.682680 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--webserver_interface=127.0.198.254
--webserver_port=34815
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:32.050894 16142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:32.051231 16142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:32.051461 16142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:32.059244 16142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:32.059404 16142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:32.059456 16142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:32.059558 16142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:32.059629 16142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:32.070230 16142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:44765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=34815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:32.071800 16142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:32.073221 16142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:32.102191 16150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.102212 16152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.102372 16149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:32.103616 16142 server_base.cc:1034] running on GCE node
I20260807 19:47:32.104719 16142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:32.106609 16142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:32.107894 16142 hybrid_clock.cc:648] HybridClock initialized: now 1786132052107844 us; error 50 us; skew 500 ppm
I20260807 19:47:32.108363 16142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:32 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:32.112541 16142 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:32.114687 16142 webserver.cc:460] Webserver started at http://127.0.198.254:34815/ using document root <none> and password file <none>
I20260807 19:47:32.115720 16142 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:32.115906 16142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:32.122663 16142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:47:32.126276 16160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:32.128100 16142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:32.128336 16142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "16852ead1388436db85f4781562507d3"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-btw8"
server_key: "53e97249c35828fb79595ca9685f2507"
server_key_iv: "faef0a84eb57ce0f5b51c50a9d9e4cec"
server_key_version: "encryptionkey@0"
I20260807 19:47:32.129546 16142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:32.143056 16142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:32.147948 16142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:32.148274 16142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:32.169243 16142 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:44765
I20260807 19:47:32.169287 16217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:44765 every 8 connection(s)
I20260807 19:47:32.170981 16142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:32.177994 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16142
I20260807 19:47:32.179122 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:36075
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=40703
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:32.179207 16218 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:47:32.185350 16218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap starting.
I20260807 19:47:32.209367 16218 log.cc:824] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:32.223248 16218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:47:32.223882 16218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap complete.
I20260807 19:47:32.233881 16218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:32.234968 16218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16852ead1388436db85f4781562507d3, State: Initialized, Role: FOLLOWER
I20260807 19:47:32.235805 16218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:32.236073 16218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:32.236205 16218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:32.236409 16218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:32.240691 16218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:32.241312 16218 leader_election.cc:302] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16852ead1388436db85f4781562507d3; no voters:
I20260807 19:47:32.242041 16218 leader_election.cc:288] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:47:32.242205 16222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:32.243712 16222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 LEADER]: Becoming Leader. State: Replica: 16852ead1388436db85f4781562507d3, State: Running, Role: LEADER
I20260807 19:47:32.244397 16222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:32.244748 16218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:32.247107 16224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16852ead1388436db85f4781562507d3. Latest consensus state: current_term: 2 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:32.247581 16224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:32.248190 16223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:32.248492 16223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:32.252023 16229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:32.262379 16229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:32.264097 16229 catalog_manager.cc:1259] Loaded cluster ID: 10a14db6b7454a4e8b849127b7e1f13e
I20260807 19:47:32.264312 16229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:32.268226 16229 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 19:47:32.287395 16229 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 19:47:32.304353 16229 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:32.306190 16229 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Loaded TSK: 0
I20260807 19:47:32.307806 16229 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:47:32.590135 16220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:32.590458 16220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:32.590617 16220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:32.598766 16220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:32.598908 16220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:32.599109 16220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:32.609786 16220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:36075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=40703
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:32.611207 16220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:32.612675 16220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:32.628885 16246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.628902 16247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.629892 16250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:32.630463 16220 server_base.cc:1034] running on GCE node
I20260807 19:47:32.631214 16220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:32.632385 16220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:32.633704 16220 hybrid_clock.cc:648] HybridClock initialized: now 1786132052633616 us; error 63 us; skew 500 ppm
I20260807 19:47:32.634223 16220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:32 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:32.638065 16220 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:32.640043 16220 webserver.cc:460] Webserver started at http://127.0.198.193:40703/ using document root <none> and password file <none>
I20260807 19:47:32.640887 16220 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:32.641088 16220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:32.647830 16220 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 19:47:32.651257 16257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:32.652995 16220 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:32.653223 16220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "6678989673b945e89b0f188882e2bdbc"
format_stamp: "Formatted at 2026-08-07 19:47:29 on dist-test-slave-btw8"
server_key: "deaea9e29f73c69fe16a20e7d7e729b3"
server_key_iv: "a6c868d7da325c1c5aad4824b154ebc2"
server_key_version: "encryptionkey@0"
I20260807 19:47:32.654346 16220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:32.677891 16220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:32.683523 16220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:32.683949 16220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:32.685322 16220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:32.687453 16220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:32.687593 16220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:47:32.687708 16220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:32.687772 16220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:32.728323 16220 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:36075
I20260807 19:47:32.728425 16379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:36075 every 8 connection(s)
I20260807 19:47:32.729933 16220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:32.736792 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16220
I20260807 19:47:32.737555 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:43533
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=33463
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:32 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:32.757808 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:32.733049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:45211 (local address 127.0.198.254:44765)
0807 19:47:32.733849 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:32.733867 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:32.734354 (+ 487us) server_negotiation.cc:407] Connection header received
0807 19:47:32.737059 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:32.737073 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:32.737508 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:32.738030 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:32.740029 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:32.741381 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:32.743222 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:32.744046 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:32.744313 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:32.748972 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:32.748982 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:32.748999 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:32.749045 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:32.752606 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:32.753534 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:32.753542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:32.753548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:32.753774 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:32.754281 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:32.754286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:32.754288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:32.754809 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:32.755029 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:32.755933 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 19:47:32.756370 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":312,"threads_started":1}
I20260807 19:47:32.757823 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:32.733190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:45211)
0807 19:47:32.734136 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:32.734184 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:32.736308 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:32.738256 (+ 1948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:32.738272 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:32.738972 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:32.739699 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:32.739726 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:32.741626 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:32.741631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:32.742993 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:32.743011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:32.744403 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:32.744412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:32.744715 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:32.745555 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:32.745602 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:32.748597 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:32.752821 (+ 4224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:32.752832 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:32.752849 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:32.753352 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:32.753909 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:32.753915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:32.753918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:32.754132 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:32.754964 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:32.754972 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:32.755336 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 19:47:32.755709 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 19:47:32.756370 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":393,"threads_started":1}
I20260807 19:47:32.761286 16380 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:32.761711 16380 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:32.762843 16380 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
I20260807 19:47:32.765707 16179 ts_manager.cc:194] Registered new tserver with Master: 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075)
I20260807 19:47:32.767253 16179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:45211
W20260807 19:47:33.115597 16384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:33.115932 16384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:33.116094 16384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:33.124961 16384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:33.125113 16384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:33.125319 16384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:33.136044 16384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:43533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=33463
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:33.137449 16384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:33.138847 16384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:33.153491 16390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:33.153549 16393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:33.153420 16391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:33.154467 16384 server_base.cc:1034] running on GCE node
I20260807 19:47:33.155146 16384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:33.156282 16384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:33.157503 16384 hybrid_clock.cc:648] HybridClock initialized: now 1786132053157452 us; error 47 us; skew 500 ppm
I20260807 19:47:33.157876 16384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:33 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:33.161562 16384 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:47:33.163476 16384 webserver.cc:460] Webserver started at http://127.0.198.194:33463/ using document root <none> and password file <none>
I20260807 19:47:33.164242 16384 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:33.164400 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:33.170455 16384 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:47:33.173915 16400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.175627 16384 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:47:33.175834 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "a04e2f276f7d43ff82ef124ed59537f1"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "672c04cdfbb2896e08cbceb49df5979f"
server_key_iv: "33a819e7d995badb40f4072ea8d8676c"
server_key_version: "encryptionkey@0"
I20260807 19:47:33.176923 16384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:33.192085 16384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:33.196907 16384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:33.197239 16384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:33.198405 16384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:33.199944 16384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:33.200073 16384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.200219 16384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:33.200312 16384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.237450 16384 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:43533
I20260807 19:47:33.237547 16521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:43533 every 8 connection(s)
I20260807 19:47:33.239044 16384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:33.244313 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16384
I20260807 19:47:33.245230 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:46187
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=45475
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:47:33 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:33.266366 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.242393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:51239 (local address 127.0.198.254:44765)
0807 19:47:33.242613 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.242620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.244631 (+ 2011us) server_negotiation.cc:407] Connection header received
0807 19:47:33.248061 (+ 3430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.248071 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.248216 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.248353 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:33.250841 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.252196 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.254315 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.254792 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.254947 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.259775 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:33.259781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:33.259785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:33.259825 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:33.262823 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.263802 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.263808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.263812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.263961 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.264482 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.264487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.264491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.264799 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:33.264965 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.265728 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.266003 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:47:33.267503 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.243189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:51239)
0807 19:47:33.244356 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:33.244464 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:33.247545 (+ 3081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:33.248665 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:33.248679 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:33.249600 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:33.250591 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.250624 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.252403 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.252409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.254107 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.254128 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.255064 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.255072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.255403 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.256270 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:33.256296 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:33.259479 (+ 3183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:33.263020 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:33.263032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:33.263052 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:33.263656 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:33.264122 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:33.264128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:33.264131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:33.264340 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:33.264938 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:33.264947 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:33.265323 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 19:47:33.265614 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:47:33.266203 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":449,"threads_started":1}
I20260807 19:47:33.269372 16522 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:33.269759 16522 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:33.270673 16522 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
I20260807 19:47:33.272751 16179 ts_manager.cc:194] Registered new tserver with Master: a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533)
I20260807 19:47:33.273746 16179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:51239
W20260807 19:47:33.611160 16525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:33.611593 16525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:33.611773 16525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:33.619594 16525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:33.619771 16525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:33.619957 16525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:33.630853 16525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:46187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=45475
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:33.632387 16525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:33.633771 16525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:33.647987 16530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:33.647915 16531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:33.647962 16533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:33.648895 16525 server_base.cc:1034] running on GCE node
I20260807 19:47:33.649593 16525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:33.650882 16525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:33.652148 16525 hybrid_clock.cc:648] HybridClock initialized: now 1786132053652095 us; error 45 us; skew 500 ppm
I20260807 19:47:33.652554 16525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:33 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:33.656482 16525 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:33.658413 16525 webserver.cc:460] Webserver started at http://127.0.198.195:45475/ using document root <none> and password file <none>
I20260807 19:47:33.659224 16525 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:33.659478 16525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:33.665673 16525 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 19:47:33.669221 16540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.671087 16525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:47:33.671334 16525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "92655035e3d84ae49d84f6c0d537ca4c"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "0a1af2bb07cb0ccd3eda0140397012ad"
server_key_iv: "ed72ac816c6371a749bbe65d46665968"
server_key_version: "encryptionkey@0"
I20260807 19:47:33.672580 16525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:33.708554 16525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:33.713289 16525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:33.713594 16525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:33.714838 16525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:33.716398 16525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:33.716497 16525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.716624 16525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:33.716720 16525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:33.756131 16525 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:46187
I20260807 19:47:33.756251 16659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:46187 every 8 connection(s)
I20260807 19:47:33.757764 16525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:33.763149 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16525
I20260807 19:47:33.772030 16380 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
Aug 07 19:47:33 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:33.781194 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.764257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.0.198.254:44765)
0807 19:47:33.764794 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.764798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.765283 (+ 485us) server_negotiation.cc:407] Connection header received
0807 19:47:33.765508 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.765515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.765584 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.765704 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:33.767500 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.768595 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.770117 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.770647 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.770798 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.774416 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:33.774421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:33.774425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:33.774458 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:33.777942 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.778715 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.778723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.778730 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.778879 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.779313 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.779318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.779321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.779805 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:33.780135 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.780541 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.780828 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":410,"thread_start_us":282,"threads_started":1}
I20260807 19:47:33.782253 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.761035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:55197 (local address 127.0.198.254:44765)
0807 19:47:33.761282 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.761287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.762714 (+ 1427us) server_negotiation.cc:407] Connection header received
0807 19:47:33.765478 (+ 2764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.765484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.765586 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.765704 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:33.767495 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.768372 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.770118 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.770690 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.770838 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.775024 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:33.775029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:33.775032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:33.775063 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:33.778247 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.779092 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.779097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.779100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.779237 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:33.779846 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:33.779853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:33.779858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:33.780380 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:33.780515 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.781605 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.781905 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":92,"spinlock_wait_cycles":33664}
I20260807 19:47:33.783619 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.761466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:55197)
0807 19:47:33.762298 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:33.762341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:33.765144 (+ 2803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:33.766015 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:33.766026 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:33.766626 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:33.767269 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.767289 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.768593 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.768598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.769941 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.769954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.770958 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.770966 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.771239 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.772049 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:33.772083 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:33.774787 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:33.778420 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:33.778431 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:33.778452 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:33.778942 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:33.779391 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:33.779396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:33.779400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:33.779693 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:33.780593 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:33.780602 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:33.781098 (+ 496us) client_negotiation.cc:769] Sending connection context
0807 19:47:33.781485 (+ 387us) client_negotiation.cc:240] Negotiation successful
0807 19:47:33.782096 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":364,"threads_started":1}
I20260807 19:47:33.785115 16660 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44765
I20260807 19:47:33.785457 16660 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:33.786281 16660 heartbeater.cc:511] Master 127.0.198.254:44765 requested a full tablet report, sending...
I20260807 19:47:33.788030 16179 ts_manager.cc:194] Registered new tserver with Master: 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:33.788874 16179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:55197
I20260807 19:47:33.797326 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:33.805511 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.798514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.0.198.254:44765)
0807 19:47:33.798806 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.798813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.799152 (+ 339us) server_negotiation.cc:407] Connection header received
0807 19:47:33.799479 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.799483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.799696 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.799892 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:33.800389 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.801217 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.802925 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.803492 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.803662 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.804279 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:33.804850 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:33.805040 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.805106 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.805210 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 19:47:33.806674 16179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:33.814939 16179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:33.856123 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.828389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43533 (local address 127.0.0.1:56624)
0807 19:47:33.844931 (+ 16542us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:33.845005 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:33.845208 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:33.846530 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:33.846539 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:33.846580 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:33.847611 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.847637 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.850158 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.850163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.853277 (+ 3114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.853295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.855011 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.855018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.855218 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.855280 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 19:47:33.855477 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 19:47:33.855720 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16252,"thread_start_us":16125,"threads_started":1}
I20260807 19:47:33.856104 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.828562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56624 (local address 127.0.198.194:43533)
0807 19:47:33.844366 (+ 15804us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.844380 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.845462 (+ 1082us) server_negotiation.cc:407] Connection header received
0807 19:47:33.845593 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.845613 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.845877 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.846159 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:33.847883 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.849936 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.853514 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.854769 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.854952 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.855225 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.855538 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.855723 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15528,"thread_start_us":15117,"threads_started":1}
I20260807 19:47:33.857363 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.845774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.0.198.195:46187)
0807 19:47:33.847609 (+ 1835us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.847616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.849114 (+ 1498us) server_negotiation.cc:407] Connection header received
0807 19:47:33.849335 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.849342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.849553 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.849784 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:33.850600 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.851978 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.854036 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.855703 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.855841 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.856078 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.856753 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.856950 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"thread_start_us":1461,"threads_started":1}
I20260807 19:47:33.857359 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.829124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.0.198.193:36075)
0807 19:47:33.843240 (+ 14116us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:33.843251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:33.845428 (+ 2177us) server_negotiation.cc:407] Connection header received
0807 19:47:33.845545 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:33.845554 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:33.845885 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:33.846120 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:33.848135 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.849953 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.853733 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.855491 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.855675 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.855948 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:33.856783 (+ 835us) server_negotiation.cc:299] Negotiation successful
0807 19:47:33.856982 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12862,"thread_start_us":13755,"threads_started":1}
I20260807 19:47:33.857270 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.845577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46187 (local address 127.0.0.1:45382)
0807 19:47:33.848980 (+ 3403us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:33.848997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:33.849155 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:33.849803 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:33.849809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:33.849838 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:33.850391 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.850402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.852133 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.852144 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.853854 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.853869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.856257 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.856264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.856493 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.856523 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:47:33.856605 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:47:33.856832 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3280,"mutex_wait_us":69,"thread_start_us":3131,"threads_started":1}
I20260807 19:47:33.858237 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:33.828516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59678)
0807 19:47:33.844930 (+ 16414us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:33.844982 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:33.845189 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:33.846500 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:33.846512 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:33.846557 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:33.847884 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.847898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.850896 (+ 2998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.850902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.853516 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:33.853532 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:33.856161 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:33.856171 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:33.856391 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:33.856498 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 19:47:33.856593 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:47:33.857767 (+ 1174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16170,"mutex_wait_us":939,"thread_start_us":15619,"threads_started":1}
I20260807 19:47:33.862053 16589 tablet_service.cc:1467] Processing CreateTablet for tablet ce2bd18e272b4ac2babc8d081b8742de (DEFAULT_TABLE table=test-table [id=bf0ed6c693904b3b915fc84772423570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:33.862221 16309 tablet_service.cc:1467] Processing CreateTablet for tablet ce2bd18e272b4ac2babc8d081b8742de (DEFAULT_TABLE table=test-table [id=bf0ed6c693904b3b915fc84772423570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:33.862231 16449 tablet_service.cc:1467] Processing CreateTablet for tablet ce2bd18e272b4ac2babc8d081b8742de (DEFAULT_TABLE table=test-table [id=bf0ed6c693904b3b915fc84772423570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:33.863690 16589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2bd18e272b4ac2babc8d081b8742de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:33.863725 16449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2bd18e272b4ac2babc8d081b8742de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:33.863745 16309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2bd18e272b4ac2babc8d081b8742de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:33.878592 16679 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Bootstrap starting.
I20260807 19:47:33.882628 16681 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Bootstrap starting.
I20260807 19:47:33.883267 16679 tablet_bootstrap.cc:654] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:33.885437 16679 log.cc:824] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:33.886526 16680 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Bootstrap starting.
I20260807 19:47:33.887403 16681 tablet_bootstrap.cc:654] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:33.889420 16681 log.cc:824] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:33.889889 16680 tablet_bootstrap.cc:654] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:33.891448 16680 log.cc:824] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:33.894430 16681 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: No bootstrap required, opened a new log
I20260807 19:47:33.894793 16681 ts_tablet_manager.cc:1397] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260807 19:47:33.895255 16679 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: No bootstrap required, opened a new log
I20260807 19:47:33.895627 16679 ts_tablet_manager.cc:1397] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260807 19:47:33.895737 16680 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: No bootstrap required, opened a new log
I20260807 19:47:33.896026 16680 ts_tablet_manager.cc:1397] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 19:47:33.905435 16681 raft_consensus.cc:348] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.905999 16681 raft_consensus.cc:374] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:33.905706 16680 raft_consensus.cc:348] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.906111 16681 raft_consensus.cc:729] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6678989673b945e89b0f188882e2bdbc, State: Initialized, Role: FOLLOWER
I20260807 19:47:33.906200 16680 raft_consensus.cc:374] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:33.906302 16680 raft_consensus.cc:729] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04e2f276f7d43ff82ef124ed59537f1, State: Initialized, Role: FOLLOWER
I20260807 19:47:33.906947 16681 consensus_queue.cc:260] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.907092 16680 consensus_queue.cc:260] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.909340 16522 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
I20260807 19:47:33.909659 16681 ts_tablet_manager.cc:1428] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260807 19:47:33.909960 16680 ts_tablet_manager.cc:1428] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:47:33.915268 16679 raft_consensus.cc:348] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.915984 16679 raft_consensus.cc:374] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:33.916107 16679 raft_consensus.cc:729] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92655035e3d84ae49d84f6c0d537ca4c, State: Initialized, Role: FOLLOWER
I20260807 19:47:33.916909 16679 consensus_queue.cc:260] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:33.918143 16660 heartbeater.cc:503] Master 127.0.198.254:44765 was elected leader, sending a full tablet report...
I20260807 19:47:33.918912 16679 ts_tablet_manager.cc:1428] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Time spent starting tablet: real 0.023s user 0.011s sys 0.005s
W20260807 19:47:33.982777 16381 tablet.cc:2369] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:47:33.991750 16523 tablet.cc:2369] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:47:34.009994 16661 tablet.cc:2369] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:34.024571 16685 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:34.025010 16685 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:34.027062 16685 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:34.035274 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.027857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:36779 (local address 127.0.198.194:43533)
0807 19:47:34.028119 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.028128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.028727 (+ 599us) server_negotiation.cc:407] Connection header received
0807 19:47:34.029175 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.029183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.029265 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.029396 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:34.030022 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.030827 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.033081 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.034167 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.034370 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.034464 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.034795 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.034948 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:34.035666 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.027278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43533 (local address 127.0.198.193:36779)
0807 19:47:34.028576 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.028619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:34.028897 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:34.029424 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:34.029429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:34.029462 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:34.029844 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:34.029858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.030987 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.030993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:34.032897 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:34.032912 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.034418 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.034425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:34.034610 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.034642 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:47:34.034765 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 19:47:34.034956 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":41,"spinlock_wait_cycles":25856,"thread_start_us":1061,"threads_started":1}
I20260807 19:47:34.035707 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.027484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46187 (local address 127.0.198.193:35007)
0807 19:47:34.029188 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.029203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:34.029360 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:34.029739 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:34.029743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:34.029771 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:34.030170 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:34.030179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.031312 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.031318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:34.033167 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:34.033181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.034644 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.034651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:34.034843 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.034868 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:47:34.034991 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 19:47:34.035150 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1568,"mutex_wait_us":46,"thread_start_us":1288,"threads_started":1}
I20260807 19:47:34.035820 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.028084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:35007 (local address 127.0.198.195:46187)
0807 19:47:34.028321 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.028329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.029306 (+ 977us) server_negotiation.cc:407] Connection header received
0807 19:47:34.029525 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.029529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.029602 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.029716 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:34.030337 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.031131 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.033326 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.034386 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.034626 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.034716 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.035065 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.035217 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:47:34.036931 16471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2bd18e272b4ac2babc8d081b8742de" candidate_uuid: "6678989673b945e89b0f188882e2bdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a04e2f276f7d43ff82ef124ed59537f1" is_pre_election: true
I20260807 19:47:34.036988 16612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2bd18e272b4ac2babc8d081b8742de" candidate_uuid: "6678989673b945e89b0f188882e2bdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92655035e3d84ae49d84f6c0d537ca4c" is_pre_election: true
I20260807 19:47:34.037521 16612 raft_consensus.cc:2454] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6678989673b945e89b0f188882e2bdbc in term 0.
I20260807 19:47:34.037521 16471 raft_consensus.cc:2454] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6678989673b945e89b0f188882e2bdbc in term 0.
I20260807 19:47:34.038409 16262 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc, 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1; no voters:
I20260807 19:47:34.039033 16685 raft_consensus.cc:2781] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:34.039247 16685 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:34.039408 16685 raft_consensus.cc:3037] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:34.041774 16685 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:34.042557 16685 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 1 election: Requested vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
I20260807 19:47:34.043174 16471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2bd18e272b4ac2babc8d081b8742de" candidate_uuid: "6678989673b945e89b0f188882e2bdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a04e2f276f7d43ff82ef124ed59537f1"
I20260807 19:47:34.043279 16612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2bd18e272b4ac2babc8d081b8742de" candidate_uuid: "6678989673b945e89b0f188882e2bdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92655035e3d84ae49d84f6c0d537ca4c"
I20260807 19:47:34.043471 16471 raft_consensus.cc:3037] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:34.043605 16612 raft_consensus.cc:3037] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:34.046286 16612 raft_consensus.cc:2454] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6678989673b945e89b0f188882e2bdbc in term 1.
I20260807 19:47:34.046300 16471 raft_consensus.cc:2454] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6678989673b945e89b0f188882e2bdbc in term 1.
I20260807 19:47:34.046846 16262 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc, 92655035e3d84ae49d84f6c0d537ca4c; no voters:
I20260807 19:47:34.047266 16685 raft_consensus.cc:2781] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:34.047811 16685 raft_consensus.cc:686] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 LEADER]: Becoming Leader. State: Replica: 6678989673b945e89b0f188882e2bdbc, State: Running, Role: LEADER
I20260807 19:47:34.048345 16685 consensus_queue.cc:237] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:34.054023 16179 catalog_manager.cc:5495] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6678989673b945e89b0f188882e2bdbc (127.0.198.193). New cstate: current_term: 1 leader_uuid: "6678989673b945e89b0f188882e2bdbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:34.092646 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.085569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.0.198.193:36075)
0807 19:47:34.085785 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.085791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.085813 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:47:34.086166 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.086171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.086283 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.086451 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:34.087033 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.088232 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.090016 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.090455 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.090565 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.091237 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:34.091972 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:34.092106 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.092332 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.092426 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:47:34.102957 16471 raft_consensus.cc:1261] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Refusing update from remote peer 6678989673b945e89b0f188882e2bdbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:34.102957 16612 raft_consensus.cc:1261] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Refusing update from remote peer 6678989673b945e89b0f188882e2bdbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:34.104472 16685 consensus_queue.cc:1035] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:34.104933 16690 consensus_queue.cc:1035] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:34.137085 16178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:34.137481 16178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:34.142139 16178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Sending DeleteTablet for 3 replicas of tablet ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:34.143891 16589 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2bd18e272b4ac2babc8d081b8742de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:34 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:45382
I20260807 19:47:34.144076 16449 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2bd18e272b4ac2babc8d081b8742de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:34 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:56624
I20260807 19:47:34.144191 16309 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2bd18e272b4ac2babc8d081b8742de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:34 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:59678
I20260807 19:47:34.145180 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16220
I20260807 19:47:34.157217 16705 tablet_replica.cc:330] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: stopping tablet replica
I20260807 19:47:34.157823 16705 raft_consensus.cc:2229] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:47:34.158654 16705 raft_consensus.cc:2258] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:34.164415 16703 tablet_replica.cc:330] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: stopping tablet replica
I20260807 19:47:34.164448 16705 ts_tablet_manager.cc:1905] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:34.164945 16703 raft_consensus.cc:2229] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:34.165560 16703 raft_consensus.cc:2258] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:34.165723 16704 tablet_replica.cc:330] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: stopping tablet replica
I20260807 19:47:34.168335 16703 ts_tablet_manager.cc:1905] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:34.172144 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.146265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.0.198.193:36075)
0807 19:47:34.146501 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.146511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.146542 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:47:34.146793 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.146803 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.146922 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.147062 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:34.149483 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.151338 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.153977 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.155183 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.155446 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.161697 (+ 6251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:34.161726 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:34.161754 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:34.161846 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:34.167228 (+ 5382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.168450 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.168460 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.168463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.168728 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.169267 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.169272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.169275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.169782 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:34.171289 (+ 1507us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.171506 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.171768 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:47:34.173079 16373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:34.174002 16704 raft_consensus.cc:2229] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:34.174659 16704 raft_consensus.cc:2258] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:34.176584 16703 ts_tablet_manager.cc:1918] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:34.176795 16703 log.cc:1196] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:34.177387 16704 ts_tablet_manager.cc:1905] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:34.177526 16703 ts_tablet_manager.cc:1939] T ce2bd18e272b4ac2babc8d081b8742de P 92655035e3d84ae49d84f6c0d537ca4c: Deleting consensus metadata
I20260807 19:47:34.179795 16165 catalog_manager.cc:4841] TS 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): tablet ce2bd18e272b4ac2babc8d081b8742de (table test-table [id=bf0ed6c693904b3b915fc84772423570]) successfully deleted
I20260807 19:47:34.187276 16704 ts_tablet_manager.cc:1918] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:34.187570 16704 log.cc:1196] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:34.188365 16704 ts_tablet_manager.cc:1939] T ce2bd18e272b4ac2babc8d081b8742de P a04e2f276f7d43ff82ef124ed59537f1: Deleting consensus metadata
I20260807 19:47:34.190759 16161 catalog_manager.cc:4841] TS a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): tablet ce2bd18e272b4ac2babc8d081b8742de (table test-table [id=bf0ed6c693904b3b915fc84772423570]) successfully deleted
I20260807 19:47:34.286803 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16220
W20260807 19:47:34.309131 16165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:36075 (error 108)
I20260807 19:47:34.310034 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16384
I20260807 19:47:34.311489 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.310655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59716)
0807 19:47:34.311138 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.311261 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
W20260807 19:47:34.311769 16165 catalog_manager.cc:4584] TS 6678989673b945e89b0f188882e2bdbc (127.0.198.193:36075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce2bd18e272b4ac2babc8d081b8742de: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
I20260807 19:47:34.325733 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.310885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.0.198.194:43533)
0807 19:47:34.311112 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.311119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.311144 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:47:34.311222 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.311230 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.311329 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.311469 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:34.312986 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.313743 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.315519 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.315995 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.316155 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.319413 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:34.319435 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:34.319461 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:34.319509 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:34.322580 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.323468 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.323476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.323478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.323615 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.324020 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.324025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.324030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.324459 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:34.324771 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.325112 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.325355 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:47:34.326439 16515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:34.365877 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.365315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59730)
0807 19:47:34.365569 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.365710 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 19:47:34.367237 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.366772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59732)
0807 19:47:34.366984 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.367101 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:47:34.422531 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16384
I20260807 19:47:34.428030 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.427333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59746)
0807 19:47:34.427654 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.427784 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 19:47:34.429421 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.428907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59748)
0807 19:47:34.429118 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.429238 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:47:34.443262 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16525
I20260807 19:47:34.460778 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.444162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.0.198.195:46187)
0807 19:47:34.444355 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.444360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.444379 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:34.444515 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.444519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.444640 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.444768 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:34.446396 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.447211 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.448861 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.449355 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.449483 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.452927 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:34.452947 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:34.452964 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:34.453009 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:34.456912 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.458002 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.458010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.458013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.458189 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.458682 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.458686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.458689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.459138 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:34.459493 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.460112 (+ 619us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.460388 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:47:34.461694 16655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:34.510787 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.509951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59758)
0807 19:47:34.510287 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.510444 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 19:47:34.512005 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.511599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59768)
0807 19:47:34.511818 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.511893 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 19:47:34.558346 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16525
I20260807 19:47:34.579457 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16142
I20260807 19:47:34.580895 16213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:34.704768 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.704149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:36075 (local address 127.0.0.1:59784)
0807 19:47:34.704373 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:34.704499 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:36075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 19:47:34.704828 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16142
I20260807 19:47:34.732087 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--webserver_interface=127.0.198.254
--webserver_port=34815
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:35.106709 16712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:35.107158 16712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:35.107303 16712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:35.117720 16712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:35.117851 16712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:35.117962 16712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:35.118039 16712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:35.118108 16712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:35.128731 16712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:44765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=34815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:35.130218 16712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:35.131703 16712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:35.144459 16718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:35.144475 16719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:35.144624 16721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:35.151345 16712 server_base.cc:1034] running on GCE node
I20260807 19:47:35.152437 16712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:35.154457 16712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:35.155731 16712 hybrid_clock.cc:648] HybridClock initialized: now 1786132055155687 us; error 49 us; skew 500 ppm
I20260807 19:47:35.156148 16712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:35 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:35.168970 16712 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:35.170924 16712 webserver.cc:460] Webserver started at http://127.0.198.254:34815/ using document root <none> and password file <none>
I20260807 19:47:35.171878 16712 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:35.172057 16712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:35.178328 16712 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:47:35.181639 16728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:35.183621 16712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:35.183881 16712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "16852ead1388436db85f4781562507d3"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-btw8"
server_key: "53e97249c35828fb79595ca9685f2507"
server_key_iv: "faef0a84eb57ce0f5b51c50a9d9e4cec"
server_key_version: "encryptionkey@0"
I20260807 19:47:35.185083 16712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:35.211982 16712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:35.217270 16712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:35.217643 16712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:35.239264 16712 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:44765
I20260807 19:47:35.239286 16785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:44765 every 8 connection(s)
I20260807 19:47:35.240985 16712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:35.247236 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16712
I20260807 19:47:35.248687 16786 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:47:35.249593 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:36075
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=40703
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:35.252882 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap starting.
I20260807 19:47:35.274778 16786 log.cc:824] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:35.293598 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:47:35.294287 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3: Bootstrap complete.
I20260807 19:47:35.304137 16786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:35.305162 16786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16852ead1388436db85f4781562507d3, State: Initialized, Role: FOLLOWER
I20260807 19:47:35.305884 16786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:35.306143 16786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:35.306288 16786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:35.306484 16786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:35.310664 16786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:35.311337 16786 leader_election.cc:302] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16852ead1388436db85f4781562507d3; no voters:
I20260807 19:47:35.312196 16786 leader_election.cc:288] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:47:35.312362 16790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:47:35.313623 16790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [term 3 LEADER]: Becoming Leader. State: Replica: 16852ead1388436db85f4781562507d3, State: Running, Role: LEADER
I20260807 19:47:35.314459 16790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } }
I20260807 19:47:35.314991 16786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:35.317466 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16852ead1388436db85f4781562507d3. Latest consensus state: current_term: 3 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:35.318043 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:35.318684 16791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "16852ead1388436db85f4781562507d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16852ead1388436db85f4781562507d3" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44765 } } }
I20260807 19:47:35.318992 16791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16852ead1388436db85f4781562507d3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:35.331139 16796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:35.342701 16796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:35.344516 16796 catalog_manager.cc:1259] Loaded cluster ID: 10a14db6b7454a4e8b849127b7e1f13e
I20260807 19:47:35.344744 16796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 19:47:35.347298 16796 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3aac4229ad google::LogMessage::Fail() at ??:0
@ 0x7f3aac42588a google::LogMessage::SendToLog() at ??:0
@ 0x7f3aac4224de google::LogMessage::Flush() at ??:0
@ 0x7f3aac423a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3ac1e55512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3ac1e52976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3ac1eda610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3ac1eda439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3aaf06e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3aaf07191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3aaf071799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3aaf04ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3ab23266db start_thread at ??:0
@ 0x7f3aab2bc61f clone at ??:0
W20260807 19:47:35.691226 16788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:35.691746 16788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:35.691939 16788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:35.699970 16788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:35.700109 16788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:35.700229 16788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:35.710806 16788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:36075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=40703
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:35.712324 16788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:35.713671 16788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:35.728508 16831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:35.729337 16832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:35.729713 16788 server_base.cc:1034] running on GCE node
W20260807 19:47:35.729980 16834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:35.730778 16788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:35.731978 16788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:35.733246 16788 hybrid_clock.cc:648] HybridClock initialized: now 1786132055733206 us; error 57 us; skew 500 ppm
I20260807 19:47:35.733662 16788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:35 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:35.737394 16788 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:35.739164 16788 webserver.cc:460] Webserver started at http://127.0.198.193:40703/ using document root <none> and password file <none>
I20260807 19:47:35.740015 16788 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:35.740185 16788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:35.746824 16788 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:47:35.750486 16841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:35.752391 16788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:47:35.752667 16788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "6678989673b945e89b0f188882e2bdbc"
format_stamp: "Formatted at 2026-08-07 19:47:29 on dist-test-slave-btw8"
server_key: "deaea9e29f73c69fe16a20e7d7e729b3"
server_key_iv: "a6c868d7da325c1c5aad4824b154ebc2"
server_key_version: "encryptionkey@0"
I20260807 19:47:35.753872 16788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:35.773566 16788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:35.779343 16788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:35.779781 16788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:35.781069 16788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:35.783565 16848 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:47:35.790253 16788 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:47:35.790468 16788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 19:47:35.790735 16788 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:47:35.794591 16788 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:47:35.794749 16788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260807 19:47:35.795153 16848 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Bootstrap starting.
I20260807 19:47:35.804765 16848 log.cc:824] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:35.822449 16848 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:47:35.823596 16848 tablet_bootstrap.cc:492] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Bootstrap complete.
I20260807 19:47:35.824376 16848 ts_tablet_manager.cc:1397] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.014s
I20260807 19:47:35.833310 16848 raft_consensus.cc:348] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:35.835273 16848 raft_consensus.cc:729] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6678989673b945e89b0f188882e2bdbc, State: Initialized, Role: FOLLOWER
I20260807 19:47:35.836328 16848 consensus_queue.cc:260] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:35.838490 16848 ts_tablet_manager.cc:1428] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260807 19:47:35.847543 16788 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:36075
I20260807 19:47:35.847610 16965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:36075 every 8 connection(s)
I20260807 19:47:35.849126 16788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:35.850136 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16788
I20260807 19:47:35.851004 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:43533
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=33463
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:35.855476 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:35.852730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:44077)
0807 19:47:35.853793 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:35.854127 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":416,"threads_started":1}
W20260807 19:47:35.856374 16967 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:44765 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:44765: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
I20260807 19:47:35.857570 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:35.857049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:52383)
0807 19:47:35.857277 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:35.857390 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:47:35.858914 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:35.858368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:49561)
0807 19:47:35.858580 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:35.858719 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:47:35.859465 16967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:47:36.227237 16970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:36.227698 16970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:36.227880 16970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:36.236099 16970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:36.236274 16970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:36.236459 16970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:36.247134 16970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:43533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=33463
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:36.248828 16970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:36.250512 16970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:36.266636 16975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.266695 16979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.266673 16976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:36.267786 16970 server_base.cc:1034] running on GCE node
I20260807 19:47:36.268529 16970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:36.269788 16970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:36.271067 16970 hybrid_clock.cc:648] HybridClock initialized: now 1786132056270980 us; error 66 us; skew 500 ppm
I20260807 19:47:36.271622 16970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:36 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132056, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:36.275509 16970 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:47:36.277297 16970 webserver.cc:460] Webserver started at http://127.0.198.194:33463/ using document root <none> and password file <none>
I20260807 19:47:36.278038 16970 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:36.278172 16970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:36.284793 16970 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:36.288414 16986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.290190 16970 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:47:36.290442 16970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "a04e2f276f7d43ff82ef124ed59537f1"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "672c04cdfbb2896e08cbceb49df5979f"
server_key_iv: "33a819e7d995badb40f4072ea8d8676c"
server_key_version: "encryptionkey@0"
I20260807 19:47:36.291699 16970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:36.321718 16970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:36.327140 16970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:36.327502 16970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:36.328614 16970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:36.330163 16970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:36.330255 16970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.330407 16970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:36.330503 16970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.367089 16970 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:43533
I20260807 19:47:36.367228 17106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:43533 every 8 connection(s)
I20260807 19:47:36.368911 16970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:36.375104 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.372724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:57021)
0807 19:47:36.373689 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.374000 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":397,"threads_started":1}
W20260807 19:47:36.376271 17107 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:44765 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:44765: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
I20260807 19:47:36.377475 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.377015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:36865)
0807 19:47:36.377239 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.377330 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:47:36.378305 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 16970
I20260807 19:47:36.378679 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.378254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:50443)
0807 19:47:36.378431 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.378542 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:47:36.378995 17107 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:47:36.379179 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:46187
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:44765
--webserver_port=45475
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:36.762084 17110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:36.762444 17110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:36.762557 17110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:36.771226 17110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:36.771481 17110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:36.771697 17110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:36.782226 17110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:46187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=45475
--tserver_master_addrs=127.0.198.254:44765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:36.783732 17110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:36.785159 17110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:36.800729 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.800704 17116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.800761 17119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:36.802101 17110 server_base.cc:1034] running on GCE node
I20260807 19:47:36.802820 17110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:36.803987 17110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:36.805234 17110 hybrid_clock.cc:648] HybridClock initialized: now 1786132056805169 us; error 46 us; skew 500 ppm
I20260807 19:47:36.805594 17110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:36 dist-test-slave-btw8 krb5kdc[15541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132056, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:36.809041 17110 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:36.810827 17110 webserver.cc:460] Webserver started at http://127.0.198.195:45475/ using document root <none> and password file <none>
I20260807 19:47:36.811707 17110 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:36.811864 17110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:36.818176 17110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:47:36.821686 17127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.823550 17110 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:36.823786 17110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "92655035e3d84ae49d84f6c0d537ca4c"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-btw8"
server_key: "0a1af2bb07cb0ccd3eda0140397012ad"
server_key_iv: "ed72ac816c6371a749bbe65d46665968"
server_key_version: "encryptionkey@0"
I20260807 19:47:36.825034 17110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:36.848564 17110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:36.853914 17110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:36.854241 17110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:36.855479 17110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:36.857123 17110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:36.857262 17110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.857410 17110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:36.857502 17110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.861131 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.860141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:37913)
0807 19:47:36.860674 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.860822 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":226,"threads_started":1}
I20260807 19:47:36.896889 17110 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:46187
I20260807 19:47:36.896960 17247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:46187 every 8 connection(s)
I20260807 19:47:36.898617 17110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:36.904250 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.901915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:45549)
0807 19:47:36.902772 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.903058 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":344,"threads_started":1}
W20260807 19:47:36.905042 17248 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:44765 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:44765: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
I20260807 19:47:36.906065 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.905655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:36807)
0807 19:47:36.905848 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.905944 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:47:36.906265 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17110
I20260807 19:47:36.907162 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.906780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:60507)
0807 19:47:36.906940 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:36.907035 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:47:36.907536 17248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:47:37.149282 17252 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:37.149711 17252 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:37.151701 17252 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
Aug 07 19:47:37 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
Aug 07 19:47:37 dist-test-slave-btw8 krb5kdc[15541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:47:37.177888 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.152070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46187 (local address 127.0.198.193:37677)
0807 19:47:37.154788 (+ 2718us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:37.154812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:37.154966 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:37.158203 (+ 3237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:37.158207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:37.159028 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:37.159803 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:37.159818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.161661 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.161668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:37.163437 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:37.163453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.164661 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.164669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:37.165063 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:37.166018 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:37.166061 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:37.169561 (+ 3500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:37.173725 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:37.173737 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:37.173756 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:37.174416 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:37.174923 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:37.174928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:37.174932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:37.175166 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:37.176123 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:37.176131 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:37.176508 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 19:47:37.176779 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 19:47:37.177409 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2545,"mutex_wait_us":195,"thread_start_us":2395,"threads_started":1}
I20260807 19:47:37.178223 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.152498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:56319 (local address 127.0.198.194:43533)
0807 19:47:37.154371 (+ 1873us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:37.154394 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:37.154450 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:47:37.156414 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:37.156426 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:37.156902 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:37.157236 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:37.160043 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.161421 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.163624 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.164435 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.164709 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:37.170206 (+ 5497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:37.170216 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:37.170230 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:37.170277 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:37.173615 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:37.174613 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:37.174621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:37.174627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:37.174826 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:37.175313 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:37.175325 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:37.175327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:37.175970 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:37.176194 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:37.177353 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0807 19:47:37.177787 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1688,"thread_start_us":1552,"threads_started":1}
I20260807 19:47:37.178711 17255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.152532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:37677 (local address 127.0.198.195:46187)
0807 19:47:37.155612 (+ 3080us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:37.155622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:37.155660 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:47:37.157238 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:37.157255 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:37.157718 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:37.158093 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:37.160045 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.161363 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.163616 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.164380 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.164541 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:37.169989 (+ 5448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:37.169999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:37.170016 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:37.170069 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:37.173500 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:37.174603 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:37.174612 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:37.174618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:37.174794 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:37.175307 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:37.175312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:37.175314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:37.175968 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:37.176195 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:37.177106 (+ 911us) server_negotiation.cc:299] Negotiation successful
0807 19:47:37.177609 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2908,"thread_start_us":2680,"threads_started":1}
I20260807 19:47:37.179086 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.151859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:43533 (local address 127.0.198.193:56319)
0807 19:47:37.152115 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:37.152164 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:37.152807 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:37.157508 (+ 4701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:37.157520 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:37.158524 (+ 1004us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:37.159706 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:37.159788 (+ 82us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.161679 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.161683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:37.163429 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:37.163448 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:37.164743 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:37.164751 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:37.165065 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:37.166585 (+ 1520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:37.166604 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:37.169817 (+ 3213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:37.173804 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:37.173810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:37.173814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:37.174429 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:37.174958 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:37.174964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:37.174967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:37.175179 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:37.176122 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:37.176131 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:37.176533 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 19:47:37.176803 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 19:47:37.178695 (+ 1892us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":1588,"spinlock_wait_cycles":29952}
W20260807 19:47:37.182137 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:37.182459 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:37.182667 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:37.183108 17252 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:37.380882 17256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.379826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:59259)
0807 19:47:37.380397 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:37.380603 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":259,"threads_started":1}
I20260807 19:47:37.863231 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.862288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:40111)
0807 19:47:37.862938 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:37.863044 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":300,"threads_started":1}
I20260807 19:47:37.909175 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:37.908119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:45935)
0807 19:47:37.908718 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:37.908864 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":315,"threads_started":1}
I20260807 19:47:38.383237 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:38.382282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:59805)
0807 19:47:38.382939 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:38.383057 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":357,"threads_started":1}
I20260807 19:47:38.865329 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:38.864398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:49825)
0807 19:47:38.864990 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:38.865109 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":284,"threads_started":1}
I20260807 19:47:38.911509 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:38.910456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:46349)
0807 19:47:38.911051 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:38.911172 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":332,"threads_started":1}
I20260807 19:47:39.064880 17262 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:39.065145 17262 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:39.065989 17262 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
W20260807 19:47:39.067119 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:39.067345 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:39.067514 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:39.067859 17262 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:39.385442 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:39.384546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:34627)
0807 19:47:39.385141 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:39.385247 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":301,"threads_started":1}
I20260807 19:47:39.867900 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:39.866746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:60621)
0807 19:47:39.867424 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:39.867640 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":402,"threads_started":1}
I20260807 19:47:39.913904 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:39.912847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:41805)
0807 19:47:39.913508 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:39.913677 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":329,"threads_started":1}
I20260807 19:47:40.387283 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.386482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:50907)
0807 19:47:40.387003 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.387096 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":282,"threads_started":1}
I20260807 19:47:40.870512 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.869319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:44083)
0807 19:47:40.870131 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.870273 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":435,"threads_started":1}
I20260807 19:47:40.916123 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.915101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:44709)
0807 19:47:40.915814 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.915917 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":347,"threads_started":1}
I20260807 19:47:41.352478 17269 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:41.352804 17269 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:41.353690 17269 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
W20260807 19:47:41.355074 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:41.355314 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:41.355504 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:41.355954 17269 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:41.389432 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.388514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:46883)
0807 19:47:41.389115 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.389242 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":356,"threads_started":1}
I20260807 19:47:41.872555 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.871758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:40981)
0807 19:47:41.872278 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.872371 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
I20260807 19:47:41.918361 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.917427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:49999)
0807 19:47:41.918072 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.918190 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":394,"threads_started":1}
I20260807 19:47:42.391520 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.390642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:33077)
0807 19:47:42.391141 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:42.391236 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":259,"threads_started":1}
I20260807 19:47:42.874745 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.873802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:57641)
0807 19:47:42.874374 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:42.874463 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":329,"threads_started":1}
I20260807 19:47:42.920518 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.919603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:59081)
0807 19:47:42.920174 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:42.920308 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":334,"threads_started":1}
I20260807 19:47:43.393651 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:43.392670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:56015)
0807 19:47:43.393363 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:43.393449 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":364,"threads_started":1}
I20260807 19:47:43.876852 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:43.876032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:45489)
0807 19:47:43.876534 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:43.876687 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":254,"threads_started":1}
I20260807 19:47:43.922837 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:43.921746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:34689)
0807 19:47:43.922466 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:43.922591 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":315,"threads_started":1}
I20260807 19:47:44.395787 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:44.394841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:34473)
0807 19:47:44.395405 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:44.395526 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":272,"threads_started":1}
I20260807 19:47:44.878959 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:44.878096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:56803)
0807 19:47:44.878660 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:44.878775 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":294,"threads_started":1}
I20260807 19:47:44.925011 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:44.924182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:39607)
0807 19:47:44.924694 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:44.924805 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
I20260807 19:47:45.397886 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.396926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:59423)
0807 19:47:45.397621 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.397725 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":267,"threads_started":1}
I20260807 19:47:45.822221 17284 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.822428 17284 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:45.823252 17284 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
W20260807 19:47:45.824434 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:45.824759 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:45.824930 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:45.825251 17284 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:45.881485 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.880453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:45757)
0807 19:47:45.881091 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.881237 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":331,"threads_started":1}
I20260807 19:47:45.927464 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.926398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:39325)
0807 19:47:45.927058 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.927177 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":340,"threads_started":1}
I20260807 19:47:46.400323 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.399111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:53595)
0807 19:47:46.399860 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.400009 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":423,"threads_started":1}
I20260807 19:47:46.883769 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.882833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:52557)
0807 19:47:46.883341 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.883516 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":268,"threads_started":1}
I20260807 19:47:46.929857 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.928790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:50155)
0807 19:47:46.929525 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.929629 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":489,"threads_started":1}
I20260807 19:47:47.402383 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.401484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:41567)
0807 19:47:47.402090 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:47.402203 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":285,"threads_started":1}
I20260807 19:47:47.885809 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.885008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:46023)
0807 19:47:47.885535 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:47.885627 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":298,"threads_started":1}
I20260807 19:47:47.931874 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.930984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:49809)
0807 19:47:47.931597 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:47.931696 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":277,"threads_started":1}
I20260807 19:47:48.404479 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:48.403620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:33439)
0807 19:47:48.404195 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:48.404294 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":301,"threads_started":1}
I20260807 19:47:48.887844 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:48.886950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:58673)
0807 19:47:48.887553 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:48.887655 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":298,"threads_started":1}
I20260807 19:47:48.933887 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:48.932992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:35763)
0807 19:47:48.933558 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:48.933653 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":313,"threads_started":1}
I20260807 19:47:49.129702 17296 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:49.130050 17296 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:49.131043 17296 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
W20260807 19:47:49.132287 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:49.132507 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:49.132686 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:49.133024 17296 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:49.406657 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:49.405822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:37271)
0807 19:47:49.406381 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:49.406487 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":285,"threads_started":1}
I20260807 19:47:49.889899 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:49.888964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:49343)
0807 19:47:49.889505 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:49.889690 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":288,"threads_started":1}
I20260807 19:47:49.936197 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:49.934998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:43591)
0807 19:47:49.935720 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:49.935894 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":276,"threads_started":1}
I20260807 19:47:50.408788 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:50.407896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:46243)
0807 19:47:50.408481 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:50.408571 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":322,"threads_started":1}
I20260807 19:47:50.892463 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:50.891565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:56677)
0807 19:47:50.892121 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:50.892226 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":273,"threads_started":1}
I20260807 19:47:50.938611 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:50.937424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:34541)
0807 19:47:50.938108 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:50.938222 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":363,"threads_started":1}
I20260807 19:47:51.411078 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:51.410240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.194:59273)
0807 19:47:51.410796 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:51.410915 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":295,"threads_started":1}
I20260807 19:47:51.601941 17305 raft_consensus.cc:482] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:51.602210 17305 raft_consensus.cc:504] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e2f276f7d43ff82ef124ed59537f1" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 43533 } } peers { permanent_uuid: "6678989673b945e89b0f188882e2bdbc" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 36075 } } peers { permanent_uuid: "92655035e3d84ae49d84f6c0d537ca4c" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46187 } }
I20260807 19:47:51.603112 17305 leader_election.cc:288] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533), 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187)
W20260807 19:47:51.604264 16842 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a04e2f276f7d43ff82ef124ed59537f1 (127.0.198.194:43533): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
W20260807 19:47:51.604511 16845 leader_election.cc:341] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92655035e3d84ae49d84f6c0d537ca4c (127.0.198.195:46187): Not found: Tablet not found: ce2bd18e272b4ac2babc8d081b8742de
I20260807 19:47:51.604656 16845 leader_election.cc:302] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6678989673b945e89b0f188882e2bdbc; no voters: 92655035e3d84ae49d84f6c0d537ca4c, a04e2f276f7d43ff82ef124ed59537f1
I20260807 19:47:51.605000 17305 raft_consensus.cc:2726] T ce2bd18e272b4ac2babc8d081b8742de P 6678989673b945e89b0f188882e2bdbc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:47:51.894847 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:51.893838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.193:45475)
0807 19:47:51.894456 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:51.894572 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":323,"threads_started":1}
I20260807 19:47:51.911907 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16788
I20260807 19:47:51.933121 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:51.913307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.0.198.193:36075)
0807 19:47:51.915406 (+ 2099us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:51.915420 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:51.915458 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:47:51.915551 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:51.915560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:51.915804 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:51.916017 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:51.917884 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:51.919107 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:51.920674 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:51.921381 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:51.921570 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:51.925160 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:51.925180 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:51.925198 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:51.925253 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:51.928760 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:51.930054 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:51.930073 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:51.930076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:51.930362 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:51.930837 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:51.930843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:51.930846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:51.931311 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:51.932002 (+ 691us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:51.932407 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 19:47:51.932749 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1826,"thread_start_us":1633,"threads_started":1}
I20260807 19:47:51.934461 16961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:51.941512 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:51.940297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44765 (local address 127.0.198.195:51569)
0807 19:47:51.941184 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:51.941326 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":264,"threads_started":1}
I20260807 19:47:52.034394 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16788
I20260807 19:47:52.057745 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 16970
I20260807 19:47:52.073321 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:52.058686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.0.198.194:43533)
0807 19:47:52.059274 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:52.059278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:52.059294 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:47:52.059401 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:52.059408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:52.059477 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:52.059598 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:52.061179 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:52.062053 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:52.063488 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:52.063968 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:52.064125 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:52.067113 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:52.067119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:52.067122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:52.067173 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:52.070175 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:52.070960 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:52.070968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:52.070976 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:52.071121 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:52.071683 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:52.071691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:52.071694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:52.071984 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:52.072299 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:52.072607 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 19:47:52.072857 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":296,"threads_started":1}
I20260807 19:47:52.074182 17102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:52.165117 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 16970
I20260807 19:47:52.187153 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17110
I20260807 19:47:52.202934 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:52.188393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.0.198.195:46187)
0807 19:47:52.189124 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:52.189129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:52.189146 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:47:52.189204 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:52.189208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:52.189295 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:52.189395 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:52.190954 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:52.191940 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:52.193291 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:52.193790 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:52.193907 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:52.196822 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:52.196828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:52.196831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:52.196870 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:52.199960 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:52.200857 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:52.200861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:52.200864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:52.201007 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:52.201377 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:52.201380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:52.201382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:52.201718 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:52.201950 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:52.202251 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 19:47:52.202441 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":433,"threads_started":1}
I20260807 19:47:52.203776 17242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:52.291801 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17110
2026-08-07T19:47:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25551 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:52 dist-test-slave-btw8 krb5kdc[17317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:52 dist-test-slave-btw8 krb5kdc[17317](info): set up 2 sockets
Aug 07 19:47:52 dist-test-slave-btw8 krb5kdc[17317](info): commencing operation
krb5kdc: starting...
W20260807 19:47:53.500981 795 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:47:53 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:47:53.853245 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:54.247502 17330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:54.247851 17330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:54.247953 17330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:54.255853 17330 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:47:54.255982 17330 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:47:54.256023 17330 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:47:54.256070 17330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:54.256121 17330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:54.256196 17330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:54.256245 17330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:54.256294 17330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:54.267410 17330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:54.268784 17330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:54.270124 17330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:54.283875 17340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:54.283972 17336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:54.283948 17338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:54.284518 17330 server_base.cc:1034] running on GCE node
I20260807 19:47:54.285557 17330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:54.287128 17330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:54.288376 17330 hybrid_clock.cc:648] HybridClock initialized: now 1786132074288291 us; error 64 us; skew 500 ppm
I20260807 19:47:54.288754 17330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:54 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132074, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:54.296748 17330 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:47:54.298704 17330 webserver.cc:460] Webserver started at http://127.0.198.254:41843/ using document root <none> and password file <none>
I20260807 19:47:54.299679 17330 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:54.299817 17330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:54.300190 17330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:54.303882 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "621792d66ba24fc6b203d0813e474651"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "2a8424580cbaffeb6ba5a4df483012cb"
server_key_iv: "197445c2981bc275c6917e531d97bf66"
server_key_version: "encryptionkey@0"
I20260807 19:47:54.304837 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "621792d66ba24fc6b203d0813e474651"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "2a8424580cbaffeb6ba5a4df483012cb"
server_key_iv: "197445c2981bc275c6917e531d97bf66"
server_key_version: "encryptionkey@0"
I20260807 19:47:54.311717 17330 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:47:54.316308 17348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:54.318128 17330 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:47:54.318403 17330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "621792d66ba24fc6b203d0813e474651"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "2a8424580cbaffeb6ba5a4df483012cb"
server_key_iv: "197445c2981bc275c6917e531d97bf66"
server_key_version: "encryptionkey@0"
I20260807 19:47:54.318702 17330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:54.345716 17330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:54.350966 17330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:54.351311 17330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:54.373250 17330 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33813
I20260807 19:47:54.373265 17402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33813 every 8 connection(s)
I20260807 19:47:54.374850 17330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:54.379069 17403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:54.380095 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17330
I20260807 19:47:54.380342 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:47:54.381256 795 external_mini_cluster.cc:1428] Setting key 00ae0e722690d5c1418f8ef5621a38e1
I20260807 19:47:54.395912 17403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap starting.
Aug 07 19:47:54 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:54.399328 17403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:54.400810 17403 log.cc:824] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:54.405400 17403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: No bootstrap required, opened a new log
I20260807 19:47:54.410624 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:54.384185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.0.198.254:33813)
0807 19:47:54.385326 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:54.385339 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:54.385403 (+ 64us) server_negotiation.cc:407] Connection header received
0807 19:47:54.387257 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:54.387285 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:54.387866 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:54.388263 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:54.390455 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:54.392081 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:54.394216 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:54.395069 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:54.395242 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:54.400114 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:54.400127 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:54.400148 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:54.400204 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:54.404923 (+ 4719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:54.405959 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:54.405972 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:54.405989 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:54.406256 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:54.406698 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:54.406704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:54.406707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:54.407559 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:54.407938 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:54.408422 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 19:47:54.409004 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":508,"threads_started":1}
I20260807 19:47:54.415860 17403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:47:54.416333 17403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:54.416428 17403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Initialized, Role: FOLLOWER
I20260807 19:47:54.417037 17403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:47:54.417313 17403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:54.417474 17403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:54.417665 17403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:54.420533 17403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:47:54.421094 17403 leader_election.cc:302] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 621792d66ba24fc6b203d0813e474651; no voters:
I20260807 19:47:54.421912 17403 leader_election.cc:288] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:54.422056 17408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:54.422991 17408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 LEADER]: Becoming Leader. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Running, Role: LEADER
I20260807 19:47:54.423838 17408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:47:54.424899 17403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:54.427626 17410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:47:54.428136 17410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:54.430598 17417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:54.430454 17411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 621792d66ba24fc6b203d0813e474651. Latest consensus state: current_term: 1 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:47:54.430816 17411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:54.437620 17417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:54.447808 17417 catalog_manager.cc:1347] Generated new cluster ID: 7a17befa71a04ee8a8ced9770a2bd222
I20260807 19:47:54.447962 17417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:54.462901 17417 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:54.464519 17417 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:54.471966 17417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Generated new TSK 0
I20260807 19:47:54.472950 17417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:47:54.565068 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:54.941998 17431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:54.942323 17431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:54.942427 17431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:54.950322 17431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:54.950460 17431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:54.950580 17431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:47:54.961424 17431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:54.962867 17431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:54.964313 17431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:54.979527 17436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:54.979476 17439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:54.979439 17437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:54.985605 17431 server_base.cc:1034] running on GCE node
I20260807 19:47:54.986479 17431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:54.987758 17431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:54.989030 17431 hybrid_clock.cc:648] HybridClock initialized: now 1786132074988980 us; error 45 us; skew 500 ppm
I20260807 19:47:54.989420 17431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:54 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132074, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:54.993415 17431 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:47:54.995582 17431 webserver.cc:460] Webserver started at http://127.0.198.193:41297/ using document root <none> and password file <none>
I20260807 19:47:54.996498 17431 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:54.996685 17431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:54.997077 17431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:55.000921 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "b0c8d439d041482e9f67c984cb375772"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "8cb294e21c43c7fe1decc775a522bf2c"
server_key_iv: "5581b1abb28e8867e704963175b864e1"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.001904 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "b0c8d439d041482e9f67c984cb375772"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "8cb294e21c43c7fe1decc775a522bf2c"
server_key_iv: "5581b1abb28e8867e704963175b864e1"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.008544 17431 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:47:55.013119 17447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.015069 17431 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260807 19:47:55.015326 17431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "b0c8d439d041482e9f67c984cb375772"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "8cb294e21c43c7fe1decc775a522bf2c"
server_key_iv: "5581b1abb28e8867e704963175b864e1"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.015699 17431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:55.039783 17431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:55.045123 17431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:55.045493 17431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:55.046749 17431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:55.048646 17431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:55.048733 17431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.048827 17431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:55.048873 17431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.086481 17431 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:35167
I20260807 19:47:55.086659 17567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:35167 every 8 connection(s)
I20260807 19:47:55.088001 17431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:55.092887 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17431
I20260807 19:47:55.093142 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:47:55.093813 795 external_mini_cluster.cc:1428] Setting key a698bec83669edd437c6ed5f8f089506
Aug 07 19:47:55 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132074, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:55.113610 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:55.090779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:44975 (local address 127.0.198.254:33813)
0807 19:47:55.091286 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:55.091292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:55.092342 (+ 1050us) server_negotiation.cc:407] Connection header received
0807 19:47:55.094539 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:55.094544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:55.094659 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:55.094791 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:55.097028 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.098261 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.100231 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.100953 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.101111 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:55.105375 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:55.105384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:55.105392 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:55.105443 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:55.109342 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:55.110864 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:55.110869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:55.110871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:55.111004 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:55.111641 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:55.111645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:55.111647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:55.111939 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:55.112055 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:55.113127 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0807 19:47:55.113317 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":239,"threads_started":1}
I20260807 19:47:55.115530 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:55.091344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:44975)
0807 19:47:55.092152 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:55.092198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:55.094210 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:55.095155 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:55.095167 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:55.096072 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:55.096818 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:55.096844 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.098461 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.098465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:55.100063 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:55.100079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.101210 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.101215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:55.101438 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:55.102276 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:55.102301 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:55.105078 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:55.109517 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:55.109524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:55.109540 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:55.110689 (+ 1149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:55.111134 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:55.111139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:55.111143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:55.111501 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:55.112084 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:55.112096 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:55.112659 (+ 563us) client_negotiation.cc:769] Sending connection context
0807 19:47:55.113019 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 19:47:55.113738 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":293,"threads_started":1}
I20260807 19:47:55.117565 17568 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:47:55.118042 17568 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:55.119153 17568 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:47:55.122035 17366 ts_manager.cc:194] Registered new tserver with Master: b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167)
I20260807 19:47:55.123839 17366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:44975
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:47:55.175441 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:55.550473 17574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:55.550892 17574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:55.551084 17574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:55.558825 17574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:55.559024 17574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:55.559228 17574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:47:55.569697 17574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:55.571146 17574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:55.572695 17574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:55.587276 17580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:55.587496 17579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:55.588410 17574 server_base.cc:1034] running on GCE node
W20260807 19:47:55.589051 17582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:55.590051 17574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:55.591329 17574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:55.592614 17574 hybrid_clock.cc:648] HybridClock initialized: now 1786132075592557 us; error 70 us; skew 500 ppm
I20260807 19:47:55.592932 17574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:55 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:55.597357 17574 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:47:55.599329 17574 webserver.cc:460] Webserver started at http://127.0.198.194:34009/ using document root <none> and password file <none>
I20260807 19:47:55.600212 17574 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:55.600342 17574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:55.600631 17574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:55.604051 17574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "3170b2ff799642299003ada31ba6862c"
format_stamp: "Formatted at 2026-08-07 19:47:55 on dist-test-slave-btw8"
server_key: "4ab51342744d82dda586098b4dbde77c"
server_key_iv: "04606f15796c77177dd2ff5b20d0c502"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.604902 17574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "3170b2ff799642299003ada31ba6862c"
format_stamp: "Formatted at 2026-08-07 19:47:55 on dist-test-slave-btw8"
server_key: "4ab51342744d82dda586098b4dbde77c"
server_key_iv: "04606f15796c77177dd2ff5b20d0c502"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.611150 17574 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:47:55.615937 17589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.617782 17574 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:47:55.618021 17574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "3170b2ff799642299003ada31ba6862c"
format_stamp: "Formatted at 2026-08-07 19:47:55 on dist-test-slave-btw8"
server_key: "4ab51342744d82dda586098b4dbde77c"
server_key_iv: "04606f15796c77177dd2ff5b20d0c502"
server_key_version: "encryptionkey@0"
I20260807 19:47:55.618261 17574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:55.636723 17574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:55.642362 17574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:55.642745 17574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:55.644122 17574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:55.646056 17574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:55.646162 17574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.646343 17574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:55.646410 17574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:55.688050 17574 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:42353
I20260807 19:47:55.688088 17705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:42353 every 8 connection(s)
I20260807 19:47:55.689591 17574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:55.691293 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17574
I20260807 19:47:55.691578 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:47:55.692301 795 external_mini_cluster.cc:1428] Setting key 609f39685e67a8f78fac23a16797cd56
Aug 07 19:47:55 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:55.712327 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:55.692760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:35697 (local address 127.0.198.254:33813)
0807 19:47:55.693375 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:55.693383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:55.694645 (+ 1262us) server_negotiation.cc:407] Connection header received
0807 19:47:55.696819 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:55.696825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:55.696929 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:55.697053 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:55.698831 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.699704 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.701437 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.701922 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.702072 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:55.706044 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:55.706049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:55.706052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:55.706091 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:55.708897 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:55.709732 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:55.709739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:55.709744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:55.709884 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:55.710356 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:55.710366 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:55.710368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:55.710666 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:55.710850 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:55.711646 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 19:47:55.711911 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":314,"threads_started":1}
I20260807 19:47:55.713517 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:55.693493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:35697)
0807 19:47:55.694416 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:55.694504 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:55.696489 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:55.697338 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:55.697348 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:55.697969 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:55.698615 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:55.698636 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.699882 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.699886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:55.701271 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:55.701286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:55.702187 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:55.702194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:55.702458 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:55.703260 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:55.703284 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:55.705745 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:55.709087 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:55.709098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:55.709118 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:55.709591 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:55.710004 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:55.710010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:55.710014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:55.710229 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:55.710820 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:55.710829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:55.711215 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 19:47:55.711561 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 19:47:55.712170 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":384,"threads_started":1}
I20260807 19:47:55.715201 17706 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:47:55.715652 17706 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:55.716771 17706 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:47:55.718921 17366 ts_manager.cc:194] Registered new tserver with Master: 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353)
I20260807 19:47:55.720105 17366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:35697
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:47:55.773198 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:56.128719 17568 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
W20260807 19:47:56.159307 17714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:56.159719 17714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:56.159880 17714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:56.167521 17714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:56.167658 17714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:56.167886 17714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:47:56.178576 17714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:47:56.180076 17714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:56.181406 17714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:56.195940 17719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:56.196022 17722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:56.195981 17720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:56.196992 17714 server_base.cc:1034] running on GCE node
I20260807 19:47:56.197683 17714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:56.198746 17714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:56.200018 17714 hybrid_clock.cc:648] HybridClock initialized: now 1786132076199957 us; error 48 us; skew 500 ppm
I20260807 19:47:56.200400 17714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:56 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:56.204011 17714 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:47:56.205888 17714 webserver.cc:460] Webserver started at http://127.0.198.195:45823/ using document root <none> and password file <none>
I20260807 19:47:56.206749 17714 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:56.206923 17714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:56.207311 17714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:56.210649 17714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "3b8f1af05c5d48208e99e337e39c8519"
format_stamp: "Formatted at 2026-08-07 19:47:56 on dist-test-slave-btw8"
server_key: "c80ee5a867f0b16c82e70ecb8f486b9d"
server_key_iv: "ffed117b3f3d4ba9e85140e145535bb4"
server_key_version: "encryptionkey@0"
I20260807 19:47:56.211627 17714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "3b8f1af05c5d48208e99e337e39c8519"
format_stamp: "Formatted at 2026-08-07 19:47:56 on dist-test-slave-btw8"
server_key: "c80ee5a867f0b16c82e70ecb8f486b9d"
server_key_iv: "ffed117b3f3d4ba9e85140e145535bb4"
server_key_version: "encryptionkey@0"
I20260807 19:47:56.218150 17714 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:47:56.222659 17730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:56.224743 17714 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 19:47:56.224915 17714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "3b8f1af05c5d48208e99e337e39c8519"
format_stamp: "Formatted at 2026-08-07 19:47:56 on dist-test-slave-btw8"
server_key: "c80ee5a867f0b16c82e70ecb8f486b9d"
server_key_iv: "ffed117b3f3d4ba9e85140e145535bb4"
server_key_version: "encryptionkey@0"
I20260807 19:47:56.225170 17714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:56.262072 17714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:56.267613 17714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:56.267987 17714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:56.269179 17714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:56.270730 17714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:56.270814 17714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:56.270923 17714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:56.270977 17714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:56.310621 17714 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:42001
I20260807 19:47:56.310738 17849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:42001 every 8 connection(s)
I20260807 19:47:56.312196 17714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:56.319116 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17714
I20260807 19:47:56.319411 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:47:56.320168 795 external_mini_cluster.cc:1428] Setting key e224cf824dda9b46a8cd24e1a56241b7
Aug 07 19:47:56 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:47:56.337620 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.314971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:36051 (local address 127.0.198.254:33813)
0807 19:47:56.315620 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.315627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.316618 (+ 991us) server_negotiation.cc:407] Connection header received
0807 19:47:56.318917 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.318924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.319057 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.319200 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:56.321695 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.322848 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.325322 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.325835 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.325997 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.330136 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:56.330142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:56.330145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:56.330185 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:56.333446 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.334536 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.334542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.334545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.334699 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.335296 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.335300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.335306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.335781 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:56.335960 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.336889 (+ 929us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.337161 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":352,"threads_started":1}
I20260807 19:47:56.338680 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.315615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:36051)
0807 19:47:56.316412 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.316457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.318556 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.319664 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.319678 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.320599 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:56.321476 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.321508 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.323033 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.323037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.325129 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.325145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.326149 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.326176 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.326409 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.327187 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:56.327213 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:56.329827 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:56.333696 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:56.333707 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:56.333753 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:56.334362 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:56.334832 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:56.334840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:56.334845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:56.335143 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:56.335922 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:56.335932 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:56.336467 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.336795 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.337437 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":305,"threads_started":1}
I20260807 19:47:56.340291 17850 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:47:56.340648 17850 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:56.341509 17850 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:47:56.343418 17366 ts_manager.cc:194] Registered new tserver with Master: 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001)
I20260807 19:47:56.344591 17366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:36051
I20260807 19:47:56.349805 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:56.369486 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.354879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.0.198.254:33813)
0807 19:47:56.355107 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.355114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.355657 (+ 543us) server_negotiation.cc:407] Connection header received
0807 19:47:56.355876 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.355880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.355962 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.356057 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:56.357697 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.358637 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.360240 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.360713 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.360818 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.363557 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:56.363563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:56.363567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:56.363607 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:56.366543 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.367339 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.367343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.367345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.367553 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.367953 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.367956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.367958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.368326 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:56.368534 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.369033 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.369246 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:47:56.374722 17366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:56.386317 17366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:56.420193 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.401582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.0.198.194:42353)
0807 19:47:56.402167 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.402181 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.407788 (+ 5607us) server_negotiation.cc:407] Connection header received
0807 19:47:56.407980 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.407992 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.408324 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.408556 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:56.410009 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.412266 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.416563 (+ 4297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.418226 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.418367 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.418599 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.419577 (+ 978us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.419817 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":311,"threads_started":1}
I20260807 19:47:56.420589 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.401011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.0.198.195:42001)
0807 19:47:56.408098 (+ 7087us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.408110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.408148 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:47:56.408261 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.408270 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.408499 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.408689 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:56.410108 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.412370 (+ 2262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.416624 (+ 4254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.418216 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.418367 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.418634 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.419658 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.419909 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6599,"thread_start_us":6435,"threads_started":1}
I20260807 19:47:56.420676 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.401062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:42353 (local address 127.0.0.1:38434)
0807 19:47:56.407636 (+ 6574us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.407659 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.407814 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.408941 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.408955 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.409011 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:56.409729 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.409753 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.412546 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.412554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.416358 (+ 3804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.416372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.418800 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.418819 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.419168 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.419242 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.419482 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.419814 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6430,"mutex_wait_us":156,"thread_start_us":5249,"threads_started":1}
I20260807 19:47:56.421644 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.400993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42001 (local address 127.0.0.1:47732)
0807 19:47:56.406807 (+ 5814us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.406850 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.407038 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.409048 (+ 2010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.409055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.409087 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:56.409804 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.409824 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.412565 (+ 2741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.412570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.416391 (+ 3821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.416408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.418833 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.418839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.419186 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.419241 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.419497 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.421345 (+ 1848us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5535,"mutex_wait_us":1804,"thread_start_us":6332,"threads_started":1}
I20260807 19:47:56.423285 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.409360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:50398)
0807 19:47:56.414324 (+ 4964us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.414372 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.414563 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.415493 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.415498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.415528 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:56.416035 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.416049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.417670 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.417679 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.419817 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.419834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.422430 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.422435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.422568 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.422594 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.422718 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.422868 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4812,"thread_start_us":4553,"threads_started":1}
I20260807 19:47:56.423455 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.408308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.0.198.193:35167)
0807 19:47:56.413732 (+ 5424us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.413743 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.414526 (+ 783us) server_negotiation.cc:407] Connection header received
0807 19:47:56.414893 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.414905 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.415231 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.415530 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:56.416231 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.417511 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.420032 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.421883 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.422036 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.422287 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.422780 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.422956 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5236,"thread_start_us":5058,"threads_started":1}
I20260807 19:47:56.425498 17784 tablet_service.cc:1467] Processing CreateTablet for tablet e39153867c1e4bfab66486e570f31912 (DEFAULT_TABLE table=test-table [id=1884b3aa203f4398ba220b977fc9d596]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:56.425498 17639 tablet_service.cc:1467] Processing CreateTablet for tablet e39153867c1e4bfab66486e570f31912 (DEFAULT_TABLE table=test-table [id=1884b3aa203f4398ba220b977fc9d596]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:56.427176 17639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39153867c1e4bfab66486e570f31912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:56.427176 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39153867c1e4bfab66486e570f31912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:56.431457 17499 tablet_service.cc:1467] Processing CreateTablet for tablet e39153867c1e4bfab66486e570f31912 (DEFAULT_TABLE table=test-table [id=1884b3aa203f4398ba220b977fc9d596]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:56.433161 17499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39153867c1e4bfab66486e570f31912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:56.444132 17870 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Bootstrap starting.
I20260807 19:47:56.446636 17872 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Bootstrap starting.
I20260807 19:47:56.448200 17870 tablet_bootstrap.cc:654] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:56.449400 17870 log.cc:824] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:56.450191 17872 tablet_bootstrap.cc:654] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:56.450402 17871 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Bootstrap starting.
I20260807 19:47:56.451741 17872 log.cc:824] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:56.453634 17870 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: No bootstrap required, opened a new log
I20260807 19:47:56.453949 17870 ts_tablet_manager.cc:1397] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:47:56.454342 17871 tablet_bootstrap.cc:654] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:56.455543 17872 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: No bootstrap required, opened a new log
I20260807 19:47:56.455856 17872 ts_tablet_manager.cc:1397] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 19:47:56.455808 17871 log.cc:824] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:56.459604 17871 tablet_bootstrap.cc:492] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: No bootstrap required, opened a new log
I20260807 19:47:56.459926 17871 ts_tablet_manager.cc:1397] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260807 19:47:56.463770 17872 raft_consensus.cc:348] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.464215 17872 raft_consensus.cc:374] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:56.464322 17872 raft_consensus.cc:729] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c8d439d041482e9f67c984cb375772, State: Initialized, Role: FOLLOWER
I20260807 19:47:56.465005 17872 consensus_queue.cc:260] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.464979 17870 raft_consensus.cc:348] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.465418 17870 raft_consensus.cc:374] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:56.465524 17870 raft_consensus.cc:729] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8f1af05c5d48208e99e337e39c8519, State: Initialized, Role: FOLLOWER
I20260807 19:47:56.466267 17870 consensus_queue.cc:260] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.467092 17872 ts_tablet_manager.cc:1428] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 19:47:56.468601 17850 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
I20260807 19:47:56.469300 17870 ts_tablet_manager.cc:1428] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 19:47:56.472427 17871 raft_consensus.cc:348] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.473073 17871 raft_consensus.cc:374] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:56.473202 17871 raft_consensus.cc:729] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3170b2ff799642299003ada31ba6862c, State: Initialized, Role: FOLLOWER
I20260807 19:47:56.473927 17871 consensus_queue.cc:260] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.475402 17706 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
I20260807 19:47:56.476462 17871 ts_tablet_manager.cc:1428] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260807 19:47:56.499616 17877 raft_consensus.cc:482] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:56.499974 17877 raft_consensus.cc:504] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.502086 17877 leader_election.cc:288] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001), 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353)
I20260807 19:47:56.512987 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.502618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:43113 (local address 127.0.198.195:42001)
0807 19:47:56.502872 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.502881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.503712 (+ 831us) server_negotiation.cc:407] Connection header received
0807 19:47:56.504147 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.504155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.504259 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.504334 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:56.505556 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.506687 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.510110 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.511538 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.511739 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.511823 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.512539 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.512691 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:47:56.512969 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.503168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:53843 (local address 127.0.198.194:42353)
0807 19:47:56.503438 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.503445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.504281 (+ 836us) server_negotiation.cc:407] Connection header received
0807 19:47:56.504456 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.504460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.504528 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.504616 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:56.505415 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.506660 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.510110 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.511538 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.511739 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.511824 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.512539 (+ 715us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.512691 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:47:56.513347 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.502744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:42353 (local address 127.0.198.193:53843)
0807 19:47:56.504146 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.504164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.504288 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.504660 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.504667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.504698 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:56.505175 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.505192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.506884 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.506888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.509824 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.509846 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.511905 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.511913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.512276 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.512361 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.512475 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.512660 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"mutex_wait_us":404,"thread_start_us":1044,"threads_started":1}
I20260807 19:47:56.513372 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.502328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42001 (local address 127.0.198.193:43113)
0807 19:47:56.503579 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:56.503602 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:56.503896 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:56.504567 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:56.504570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:56.504630 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:56.505347 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.505359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.506878 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.506883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.509824 (+ 2941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:56.509846 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.511918 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.511924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:56.512279 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.512356 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 19:47:56.512475 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:47:56.512660 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"mutex_wait_us":304,"thread_start_us":1100,"threads_started":1}
I20260807 19:47:56.514577 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39153867c1e4bfab66486e570f31912" candidate_uuid: "b0c8d439d041482e9f67c984cb375772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8f1af05c5d48208e99e337e39c8519" is_pre_election: true
I20260807 19:47:56.514569 17659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39153867c1e4bfab66486e570f31912" candidate_uuid: "b0c8d439d041482e9f67c984cb375772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3170b2ff799642299003ada31ba6862c" is_pre_election: true
I20260807 19:47:56.515133 17659 raft_consensus.cc:2454] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c8d439d041482e9f67c984cb375772 in term 0.
I20260807 19:47:56.515128 17804 raft_consensus.cc:2454] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c8d439d041482e9f67c984cb375772 in term 0.
I20260807 19:47:56.515967 17451 leader_election.cc:302] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3170b2ff799642299003ada31ba6862c, b0c8d439d041482e9f67c984cb375772; no voters:
I20260807 19:47:56.516460 17877 raft_consensus.cc:2781] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:56.516601 17877 raft_consensus.cc:482] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:56.516752 17877 raft_consensus.cc:3037] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:56.519552 17877 raft_consensus.cc:504] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.520537 17877 leader_election.cc:288] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [CANDIDATE]: Term 1 election: Requested vote from peers 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001), 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353)
I20260807 19:47:56.521181 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39153867c1e4bfab66486e570f31912" candidate_uuid: "b0c8d439d041482e9f67c984cb375772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8f1af05c5d48208e99e337e39c8519"
I20260807 19:47:56.521296 17659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39153867c1e4bfab66486e570f31912" candidate_uuid: "b0c8d439d041482e9f67c984cb375772" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3170b2ff799642299003ada31ba6862c"
I20260807 19:47:56.521430 17804 raft_consensus.cc:3037] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:56.521631 17659 raft_consensus.cc:3037] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:56.524331 17804 raft_consensus.cc:2454] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c8d439d041482e9f67c984cb375772 in term 1.
I20260807 19:47:56.524899 17448 leader_election.cc:302] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8f1af05c5d48208e99e337e39c8519, b0c8d439d041482e9f67c984cb375772; no voters:
I20260807 19:47:56.525094 17659 raft_consensus.cc:2454] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c8d439d041482e9f67c984cb375772 in term 1.
I20260807 19:47:56.525279 17877 raft_consensus.cc:2781] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:56.526026 17877 raft_consensus.cc:686] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 1 LEADER]: Becoming Leader. State: Replica: b0c8d439d041482e9f67c984cb375772, State: Running, Role: LEADER
I20260807 19:47:56.526624 17877 consensus_queue.cc:237] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:47:56.531868 17365 catalog_manager.cc:5495] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0c8d439d041482e9f67c984cb375772 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "b0c8d439d041482e9f67c984cb375772" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } health_report { overall_health: HEALTHY } } }
W20260807 19:47:56.564040 17851 tablet.cc:2369] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:56.573122 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.567155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.0.198.193:35167)
0807 19:47:56.567391 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.567397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.567418 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:47:56.567606 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.567610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.567704 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.567877 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:56.568316 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.569128 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.570622 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.571049 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.571197 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.571883 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:56.572468 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:56.572644 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.572739 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.572828 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:47:56.583328 17804 raft_consensus.cc:1261] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Refusing update from remote peer b0c8d439d041482e9f67c984cb375772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:56.583341 17659 raft_consensus.cc:1261] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Refusing update from remote peer b0c8d439d041482e9f67c984cb375772: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:56.584688 17882 consensus_queue.cc:1035] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:56.585103 17877 consensus_queue.cc:1035] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:47:56.594080 17569 tablet.cc:2369] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:56.600242 17885 mvcc.cc:205] Tried to move back new op lower bound from 7315996985672859648 to 7315996985457360896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996985672859648}]
I20260807 19:47:56.602195 17886 mvcc.cc:205] Tried to move back new op lower bound from 7315996985672859648 to 7315996985457360896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996985672859648}]
I20260807 19:47:56.606117 17887 mvcc.cc:205] Tried to move back new op lower bound from 7315996985672859648 to 7315996985457360896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996985672859648}]
I20260807 19:47:56.618952 17365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:56.619282 17365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:56.624244 17365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Sending DeleteTablet for 3 replicas of tablet e39153867c1e4bfab66486e570f31912
I20260807 19:47:56.626397 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17431
I20260807 19:47:56.626458 17499 tablet_service.cc:1514] Processing DeleteTablet for tablet e39153867c1e4bfab66486e570f31912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:56 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:50398
I20260807 19:47:56.626528 17639 tablet_service.cc:1514] Processing DeleteTablet for tablet e39153867c1e4bfab66486e570f31912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:56 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:38434
I20260807 19:47:56.627548 17784 tablet_service.cc:1514] Processing DeleteTablet for tablet e39153867c1e4bfab66486e570f31912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:56 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:47732
I20260807 19:47:56.627797 17895 tablet_replica.cc:330] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: stopping tablet replica
I20260807 19:47:56.628363 17895 raft_consensus.cc:2229] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:56.629253 17895 raft_consensus.cc:2258] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:56.632766 17895 ts_tablet_manager.cc:1905] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:56.637708 17897 tablet_replica.cc:330] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: stopping tablet replica
I20260807 19:47:56.638541 17897 raft_consensus.cc:2229] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:56.639245 17897 raft_consensus.cc:2258] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:56.640393 17894 tablet_replica.cc:330] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: stopping tablet replica
I20260807 19:47:56.640923 17894 raft_consensus.cc:2229] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:47:56.641785 17894 raft_consensus.cc:2258] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:56.644439 17897 ts_tablet_manager.cc:1905] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:56.649819 17894 ts_tablet_manager.cc:1905] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:56.652972 17897 ts_tablet_manager.cc:1918] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:56.653304 17897 log.cc:1196] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/e39153867c1e4bfab66486e570f31912
I20260807 19:47:56.654058 17897 ts_tablet_manager.cc:1939] T e39153867c1e4bfab66486e570f31912 P 3b8f1af05c5d48208e99e337e39c8519: Deleting consensus metadata
I20260807 19:47:56.656641 17349 catalog_manager.cc:4841] TS 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001): tablet e39153867c1e4bfab66486e570f31912 (table test-table [id=1884b3aa203f4398ba220b977fc9d596]) successfully deleted
I20260807 19:47:56.657599 17895 ts_tablet_manager.cc:1918] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:56.657828 17895 log.cc:1196] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/e39153867c1e4bfab66486e570f31912
I20260807 19:47:56.658004 17894 ts_tablet_manager.cc:1918] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:56.658211 17894 log.cc:1196] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/e39153867c1e4bfab66486e570f31912
I20260807 19:47:56.658483 17895 ts_tablet_manager.cc:1939] T e39153867c1e4bfab66486e570f31912 P 3170b2ff799642299003ada31ba6862c: Deleting consensus metadata
I20260807 19:47:56.658802 17894 ts_tablet_manager.cc:1939] T e39153867c1e4bfab66486e570f31912 P b0c8d439d041482e9f67c984cb375772: Deleting consensus metadata
Aug 07 19:47:56 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:47:56.660578 17352 catalog_manager.cc:4841] TS 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353): tablet e39153867c1e4bfab66486e570f31912 (table test-table [id=1884b3aa203f4398ba220b977fc9d596]) successfully deleted
I20260807 19:47:56.660635 17349 catalog_manager.cc:4841] TS b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167): tablet e39153867c1e4bfab66486e570f31912 (table test-table [id=1884b3aa203f4398ba220b977fc9d596]) successfully deleted
I20260807 19:47:56.668963 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.627755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.0.198.193:35167)
0807 19:47:56.627952 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.627959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.634347 (+ 6388us) server_negotiation.cc:407] Connection header received
0807 19:47:56.640490 (+ 6143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.640496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.640639 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.640737 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:56.643459 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.644862 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.649336 (+ 4474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.650209 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.650363 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.661979 (+ 11616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:56.661996 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:56.662010 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:56.662057 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:56.665514 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.666428 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.666436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.666439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.666613 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.667052 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.667058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.667062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.667575 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:56.667938 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.668305 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.668579 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:47:56.669961 17562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:56.790200 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17431
I20260807 19:47:56.812096 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17574
Aug 07 19:47:56 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:47:56.830462 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.813034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.0.198.194:42353)
0807 19:47:56.813232 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.813237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.813257 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:47:56.813478 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.813483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.813564 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.813683 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:56.815252 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.816165 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.817829 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.818224 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.818324 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.823798 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:56.823818 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:56.823835 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:56.823881 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:56.827244 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.828114 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.828122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.828124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.828263 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.828733 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.828738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.828740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.829240 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:56.829489 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.829878 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.830111 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:47:56.831140 17701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:56.934325 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17574
I20260807 19:47:56.954835 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17714
Aug 07 19:47:56 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:47:56.974990 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:56.955852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.0.198.195:42001)
0807 19:47:56.956089 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:56.956096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:56.956144 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:47:56.956353 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:56.956357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:56.956443 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:56.956567 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:56.958193 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.959017 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.960816 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:56.961320 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:56.961497 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:56.967332 (+ 5835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:56.967475 (+ 143us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:56.967495 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:56.967547 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:56.971017 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.972154 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.972164 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.972167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.972354 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:56.972790 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:56.972794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:56.972796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:56.973234 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:56.973501 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:56.974133 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 19:47:56.974686 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:47:56.975833 17844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:57.070063 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17714
I20260807 19:47:57.089927 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17330
I20260807 19:47:57.091110 17398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:57.206238 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17330
I20260807 19:48:17.233080 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--webserver_interface=127.0.198.254
--webserver_port=41843
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:48:17.626192 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:17.626520 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:17.626688 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:17.634413 17902 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:48:17.634534 17902 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:48:17.634613 17902 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:48:17.634697 17902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:48:17.634806 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:17.634931 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:17.635020 17902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:48:17.635105 17902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:48:17.646093 17902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=41843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:17.647614 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:17.649175 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:17.679693 17909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:17.680037 17911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:17.679677 17908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:17.680274 17902 server_base.cc:1034] running on GCE node
I20260807 19:48:17.681833 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:17.684001 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:17.685344 17902 hybrid_clock.cc:648] HybridClock initialized: now 1786132097685269 us; error 61 us; skew 500 ppm
I20260807 19:48:17.685958 17902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:17 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:17.694149 17902 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:48:17.696070 17902 webserver.cc:460] Webserver started at http://127.0.198.254:41843/ using document root <none> and password file <none>
I20260807 19:48:17.696941 17902 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:17.697103 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:17.703567 17902 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:48:17.706956 17920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:17.708775 17902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:48:17.709019 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "621792d66ba24fc6b203d0813e474651"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "2a8424580cbaffeb6ba5a4df483012cb"
server_key_iv: "197445c2981bc275c6917e531d97bf66"
server_key_version: "encryptionkey@0"
I20260807 19:48:17.710242 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:17.736857 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:17.742446 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:17.742835 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:17.764984 17902 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33813
I20260807 19:48:17.765038 17971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33813 every 8 connection(s)
I20260807 19:48:17.767166 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:48:17.773744 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17902
I20260807 19:48:17.774767 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:35167
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=41297
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:48:17.776407 17972 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:48:17.781054 17972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap starting.
I20260807 19:48:17.805354 17972 log.cc:824] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Log is configured to *not* fsync() on all Append() calls
I20260807 19:48:17.820205 17972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:17.820875 17972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap complete.
I20260807 19:48:17.832158 17972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:17.833252 17972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Initialized, Role: FOLLOWER
I20260807 19:48:17.834017 17972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:17.834333 17972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:48:17.834492 17972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:48:17.834705 17972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:17.839767 17972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:17.840428 17972 leader_election.cc:302] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 621792d66ba24fc6b203d0813e474651; no voters:
I20260807 19:48:17.841236 17972 leader_election.cc:288] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:48:17.841395 17976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:17.842841 17976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 LEADER]: Becoming Leader. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Running, Role: LEADER
I20260807 19:48:17.843700 17976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:17.844135 17972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:48:17.846872 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 621792d66ba24fc6b203d0813e474651. Latest consensus state: current_term: 2 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:48:17.847378 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:17.846913 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:48:17.847527 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:17.855373 17982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:48:17.866289 17982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:48:17.868495 17982 catalog_manager.cc:1259] Loaded cluster ID: 7a17befa71a04ee8a8ced9770a2bd222
I20260807 19:48:17.868708 17982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:48:17.875552 17982 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:48:17.877974 17982 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Loaded TSK: 0
I20260807 19:48:17.916081 17982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Generated new TSK 1
I20260807 19:48:17.916538 17982 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:48:18.194823 17974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:18.195243 17974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:18.195482 17974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:18.204265 17974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:18.204452 17974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:18.204643 17974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:48:18.216423 17974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:35167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=41297
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:18.217990 17974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:18.219585 17974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:18.235492 18003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:18.236195 18002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:18.237059 17974 server_base.cc:1034] running on GCE node
W20260807 19:48:18.237234 18005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:18.238031 17974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:18.239218 17974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:18.240502 17974 hybrid_clock.cc:648] HybridClock initialized: now 1786132098240431 us; error 72 us; skew 500 ppm
I20260807 19:48:18.240878 17974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:18 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:18.244693 17974 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:48:18.246628 17974 webserver.cc:460] Webserver started at http://127.0.198.193:41297/ using document root <none> and password file <none>
I20260807 19:48:18.247517 17974 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:18.247701 17974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:18.254165 17974 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:48:18.257543 18012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.259222 17974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:48:18.259557 17974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "b0c8d439d041482e9f67c984cb375772"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "8cb294e21c43c7fe1decc775a522bf2c"
server_key_iv: "5581b1abb28e8867e704963175b864e1"
server_key_version: "encryptionkey@0"
I20260807 19:48:18.260702 17974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:18.284171 17974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:18.289527 17974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:18.289894 17974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:18.291184 17974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:18.292805 17974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:18.292922 17974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.293092 17974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:18.293201 17974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.334347 17974 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:35167
I20260807 19:48:18.334530 18130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:35167 every 8 connection(s)
I20260807 19:48:18.336167 17974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:48:18.341729 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 17974
I20260807 19:48:18.342494 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:42353
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=34009
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:48:18 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:48:18.372149 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:18.339693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:51121)
0807 19:48:18.340587 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:18.340635 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:18.342547 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:18.344754 (+ 2207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:18.344783 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:18.345415 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:48:18.346785 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:18.346818 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.349791 (+ 2973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.349802 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:18.356282 (+ 6480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:18.356319 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.358442 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.358453 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:18.358839 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:18.359807 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:48:18.359847 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:48:18.362952 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:48:18.367192 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:18.367202 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:18.367220 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:18.367818 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:18.368357 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:18.368362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:18.368366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:18.368578 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:18.369400 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:48:18.369408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:48:18.369762 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:48:18.370039 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 19:48:18.370671 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":417,"threads_started":1}
I20260807 19:48:18.372149 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:18.339643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:51121 (local address 127.0.198.254:33813)
0807 19:48:18.340386 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:18.340407 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:18.340791 (+ 384us) server_negotiation.cc:407] Connection header received
0807 19:48:18.343330 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:18.343385 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:18.344004 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:18.344513 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:18.347108 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.349402 (+ 2294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.356621 (+ 7219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.358079 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.358266 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:18.363429 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:18.363443 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:18.363463 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:18.363511 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:18.366990 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:18.368005 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:18.368013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:18.368019 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:18.368212 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:18.368715 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:18.368720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:18.368722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:18.369231 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:18.369445 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:18.370277 (+ 832us) server_negotiation.cc:299] Negotiation successful
0807 19:48:18.370681 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":271,"threads_started":1}
I20260807 19:48:18.375145 18131 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:48:18.375617 18131 heartbeater.cc:464] Registering TS with master...
I20260807 19:48:18.376705 18131 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:48:18.379323 17937 ts_manager.cc:194] Registered new tserver with Master: b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167)
I20260807 19:48:18.381057 17937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:51121
W20260807 19:48:18.739383 18137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:18.739735 18137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:18.739950 18137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:18.748883 18137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:18.749038 18137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:18.749261 18137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:48:18.761447 18137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:42353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=34009
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:18.763007 18137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:18.764925 18137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:18.782670 18145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:18.783161 18144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:18.783209 18147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:18.789541 18137 server_base.cc:1034] running on GCE node
I20260807 19:48:18.790334 18137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:18.791617 18137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:18.792910 18137 hybrid_clock.cc:648] HybridClock initialized: now 1786132098792827 us; error 71 us; skew 500 ppm
I20260807 19:48:18.793359 18137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:18 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:18.797545 18137 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:48:18.799486 18137 webserver.cc:460] Webserver started at http://127.0.198.194:34009/ using document root <none> and password file <none>
I20260807 19:48:18.800333 18137 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:18.800478 18137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:18.807888 18137 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 19:48:18.811483 18157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.813687 18137 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:48:18.813943 18137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "3170b2ff799642299003ada31ba6862c"
format_stamp: "Formatted at 2026-08-07 19:47:55 on dist-test-slave-btw8"
server_key: "4ab51342744d82dda586098b4dbde77c"
server_key_iv: "04606f15796c77177dd2ff5b20d0c502"
server_key_version: "encryptionkey@0"
I20260807 19:48:18.815768 18137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:18.854969 18137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:18.860656 18137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:18.861294 18137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:18.862705 18137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:18.864557 18137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:18.864691 18137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.864828 18137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:18.864936 18137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:18.904891 18137 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:42353
I20260807 19:48:18.905014 18278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:42353 every 8 connection(s)
I20260807 19:48:18.906553 18137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:48:18.915308 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18137
I20260807 19:48:18.916337 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:42001
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=45823
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:48:18 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:48:18.935623 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:18.909632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:56755 (local address 127.0.198.254:33813)
0807 19:48:18.910220 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:18.910227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:18.911245 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 19:48:18.913484 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:18.913489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:18.913628 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:18.913785 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:18.916164 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.917132 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.919976 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.920660 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.920807 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:18.926247 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:18.926254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:18.926258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:18.926301 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:18.930060 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:18.931739 (+ 1679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:18.931745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:18.931748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:18.931964 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:18.932636 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:18.932640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:18.932642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:18.933039 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:18.933259 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:18.934766 (+ 1507us) server_negotiation.cc:299] Negotiation successful
0807 19:48:18.935122 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":233,"threads_started":1}
I20260807 19:48:18.936896 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:18.910184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:56755)
0807 19:48:18.911060 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:18.911103 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:18.913133 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:18.914121 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:18.914131 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:18.914812 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:48:18.915841 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:18.915874 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.917337 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.917346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:18.919761 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:18.919784 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:18.920956 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:18.920966 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:18.921364 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:18.922395 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:48:18.922436 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:48:18.925874 (+ 3438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:48:18.930444 (+ 4570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:18.930467 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:18.930513 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:18.931474 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:18.932142 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:18.932148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:18.932152 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:18.932462 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:18.933241 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:48:18.933254 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:48:18.933804 (+ 550us) client_negotiation.cc:769] Sending connection context
0807 19:48:18.934456 (+ 652us) client_negotiation.cc:240] Negotiation successful
0807 19:48:18.935449 (+ 993us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":316,"threads_started":1}
I20260807 19:48:18.938959 18279 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:48:18.939407 18279 heartbeater.cc:464] Registering TS with master...
I20260807 19:48:18.940464 18279 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:48:18.942397 17937 ts_manager.cc:194] Registered new tserver with Master: 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353)
I20260807 19:48:18.943848 17937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:56755
W20260807 19:48:19.303996 18283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:19.304477 18283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:19.304653 18283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:19.318116 18283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:19.318311 18283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:19.318490 18283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:48:19.338003 18283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:42001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=45823
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:19.340032 18283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:19.342175 18283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:19.358453 18292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:19.358541 18290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:19.358479 18289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:19.359340 18283 server_base.cc:1034] running on GCE node
I20260807 19:48:19.360086 18283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:19.361231 18283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:19.362493 18283 hybrid_clock.cc:648] HybridClock initialized: now 1786132099362442 us; error 42 us; skew 500 ppm
I20260807 19:48:19.363001 18283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:19 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:19.367331 18283 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:48:19.369460 18283 webserver.cc:460] Webserver started at http://127.0.198.195:45823/ using document root <none> and password file <none>
I20260807 19:48:19.370321 18283 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:19.370519 18283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:19.376794 18283 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:48:19.379984 18301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:19.381532 18283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:48:19.381760 18283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "3b8f1af05c5d48208e99e337e39c8519"
format_stamp: "Formatted at 2026-08-07 19:47:56 on dist-test-slave-btw8"
server_key: "c80ee5a867f0b16c82e70ecb8f486b9d"
server_key_iv: "ffed117b3f3d4ba9e85140e145535bb4"
server_key_version: "encryptionkey@0"
I20260807 19:48:19.382962 18283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:19.385380 18131 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
I20260807 19:48:19.399456 18283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:19.404244 18283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:19.404503 18283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:19.405529 18283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:19.406908 18283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:19.407038 18283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:19.407128 18283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:19.407186 18283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:19.445998 18283 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:42001
I20260807 19:48:19.446062 18423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:42001 every 8 connection(s)
I20260807 19:48:19.447780 18283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:48:19.457261 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18283
Aug 07 19:48:19 dist-test-slave-btw8 krb5kdc[17317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:48:19.475010 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.450778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:59877 (local address 127.0.198.254:33813)
0807 19:48:19.451476 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.451482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.452345 (+ 863us) server_negotiation.cc:407] Connection header received
0807 19:48:19.454370 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.454375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.454466 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.454580 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:19.456594 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.457425 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.459843 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.460634 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.460788 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.465756 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:19.465762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:19.465767 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:19.465856 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:19.470071 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.471452 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.471459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.471464 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.471703 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.472343 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.472348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.472350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.472805 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:19.472964 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.474030 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.474340 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":346,"threads_started":1}
I20260807 19:48:19.475754 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.451153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:59877)
0807 19:48:19.452140 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.452199 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.454042 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.454861 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.454870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.455691 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:48:19.456394 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.456415 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.457657 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.457727 (+ 70us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.459650 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.459676 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.460901 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.460907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.461127 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.462061 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:48:19.462100 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:48:19.465379 (+ 3279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:48:19.470331 (+ 4952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:19.470344 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:19.470369 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:19.471241 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:19.471853 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:48:19.471862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:48:19.471867 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:48:19.472153 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:48:19.472964 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:48:19.472977 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:48:19.473492 (+ 515us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.473853 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.474450 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":446,"threads_started":1}
I20260807 19:48:19.477687 18424 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33813
I20260807 19:48:19.478071 18424 heartbeater.cc:464] Registering TS with master...
I20260807 19:48:19.478092 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.458477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.0.198.254:33813)
0807 19:48:19.459049 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.459055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.459455 (+ 400us) server_negotiation.cc:407] Connection header received
0807 19:48:19.459631 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.459636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.459722 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.459834 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:19.462021 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.463230 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.465408 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.466045 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.466232 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.470380 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:19.470390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:19.470395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:19.470436 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:19.474171 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.475577 (+ 1406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.475582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.475586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.475734 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.476376 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.476383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.476387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.476773 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:19.477059 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.477464 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.477733 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260807 19:48:19.479019 18424 heartbeater.cc:511] Master 127.0.198.254:33813 requested a full tablet report, sending...
I20260807 19:48:19.480799 17936 ts_manager.cc:194] Registered new tserver with Master: 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001)
I20260807 19:48:19.481779 17936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:59877
I20260807 19:48:19.494155 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:48:19.502549 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.495320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45532 (local address 127.0.198.254:33813)
0807 19:48:19.495783 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.495790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.496208 (+ 418us) server_negotiation.cc:407] Connection header received
0807 19:48:19.496469 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.496473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.496685 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.496824 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:48:19.497304 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.498149 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.500256 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.500762 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.500878 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.501550 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:48:19.501916 (+ 366us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 19:48:19.502257 (+ 341us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45532: token expired
Metrics: {"server-negotiator.queue_time_us":185}
W20260807 19:48:19.502736 18428 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45532: token expired
W20260807 19:48:19.503525 17854 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.198.254:33813)
I20260807 19:48:19.518482 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.504038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.0.198.254:33813)
0807 19:48:19.504237 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.504241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.504260 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:48:19.504377 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.504389 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.504452 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.504535 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:19.506188 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.506944 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.508653 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.509082 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.509185 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.512553 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:19.512559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:19.512562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:19.512601 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:19.515648 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.516437 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.516441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.516444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.516581 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.517046 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.517049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.517052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.517410 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:19.517615 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.517995 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.518180 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:48:19.522686 17936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:48:19.530424 17936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:48:19.561290 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.544655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:42353 (local address 127.0.0.1:47336)
0807 19:48:19.546759 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.546816 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.547016 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.548769 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.548781 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.548825 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:48:19.549480 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.549495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.553972 (+ 4477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.553982 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.557556 (+ 3574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.557589 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.560080 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.560096 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.560368 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.560446 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.560574 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.560812 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1790,"thread_start_us":1561,"threads_started":1}
I20260807 19:48:19.562211 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.545436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.0.198.194:42353)
0807 19:48:19.547873 (+ 2437us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.547883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.547914 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:48:19.548039 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.548052 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.548399 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.549763 (+ 1364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:48:19.550071 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.553728 (+ 3657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.557952 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.559769 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.561255 (+ 1486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.561532 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.561624 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.561854 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2267,"thread_start_us":2595,"threads_started":1}
I20260807 19:48:19.565248 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.544930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42001 (local address 127.0.0.1:44024)
0807 19:48:19.547775 (+ 2845us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.547793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.547968 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.551480 (+ 3512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.551490 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.551554 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:48:19.552264 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.552286 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.559567 (+ 7281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.559572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.561901 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.561920 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.564153 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.564163 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.564355 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.564397 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.564514 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.564717 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2704,"thread_start_us":2509,"threads_started":1}
I20260807 19:48:19.566128 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.545621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.0.198.195:42001)
0807 19:48:19.547636 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.547648 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.549910 (+ 2262us) server_negotiation.cc:407] Connection header received
0807 19:48:19.550241 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.550257 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.550641 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.550932 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:48:19.554820 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.559334 (+ 4514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.562165 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.563468 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.563634 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.563884 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.565448 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.565709 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1793,"thread_start_us":1526,"threads_started":1}
I20260807 19:48:19.567019 18208 tablet_service.cc:1467] Processing CreateTablet for tablet e93698ac442e442bae0e5fafd444e084 (DEFAULT_TABLE table=test-table [id=721e046404c74f8e9cb2a516328e2983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:48:19.569357 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.547915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58410)
0807 19:48:19.558577 (+ 10662us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.558609 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.558756 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.560845 (+ 2089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.560850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.560882 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:48:19.561375 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.561390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.563602 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.563607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.566331 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.566368 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.568484 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.568489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.568626 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.568652 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.568787 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.568941 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10458,"thread_start_us":10932,"threads_started":1}
I20260807 19:48:19.569381 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.548448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.0.198.193:35167)
0807 19:48:19.560243 (+ 11795us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.560251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.560280 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:48:19.560371 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.560378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.560599 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.560880 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:48:19.561614 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.563395 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.566560 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.568215 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.568399 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.568661 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.568858 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.569027 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11592,"thread_start_us":11427,"threads_started":1}
I20260807 19:48:19.570369 18208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93698ac442e442bae0e5fafd444e084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:19.570369 18353 tablet_service.cc:1467] Processing CreateTablet for tablet e93698ac442e442bae0e5fafd444e084 (DEFAULT_TABLE table=test-table [id=721e046404c74f8e9cb2a516328e2983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:48:19.572440 18353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93698ac442e442bae0e5fafd444e084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:19.573757 18062 tablet_service.cc:1467] Processing CreateTablet for tablet e93698ac442e442bae0e5fafd444e084 (DEFAULT_TABLE table=test-table [id=721e046404c74f8e9cb2a516328e2983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:48:19.575309 18062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93698ac442e442bae0e5fafd444e084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:19.584486 18446 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Bootstrap starting.
I20260807 19:48:19.586907 18447 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Bootstrap starting.
I20260807 19:48:19.589689 18446 tablet_bootstrap.cc:654] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:19.590515 18447 tablet_bootstrap.cc:654] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:19.591159 18446 log.cc:824] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Log is configured to *not* fsync() on all Append() calls
I20260807 19:48:19.591897 18447 log.cc:824] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:48:19.591960 18448 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Bootstrap starting.
I20260807 19:48:19.596159 18447 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: No bootstrap required, opened a new log
I20260807 19:48:19.596221 18448 tablet_bootstrap.cc:654] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:19.596344 18446 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: No bootstrap required, opened a new log
I20260807 19:48:19.596530 18447 ts_tablet_manager.cc:1397] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260807 19:48:19.596653 18446 ts_tablet_manager.cc:1397] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.001s
I20260807 19:48:19.597620 18448 log.cc:824] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Log is configured to *not* fsync() on all Append() calls
I20260807 19:48:19.602862 18448 tablet_bootstrap.cc:492] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: No bootstrap required, opened a new log
I20260807 19:48:19.603222 18448 ts_tablet_manager.cc:1397] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 19:48:19.605486 18447 raft_consensus.cc:348] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.605926 18447 raft_consensus.cc:374] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:19.606068 18447 raft_consensus.cc:729] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3170b2ff799642299003ada31ba6862c, State: Initialized, Role: FOLLOWER
I20260807 19:48:19.606770 18447 consensus_queue.cc:260] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.607609 18446 raft_consensus.cc:348] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.608067 18446 raft_consensus.cc:374] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:19.608109 18279 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
I20260807 19:48:19.608189 18446 raft_consensus.cc:729] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8f1af05c5d48208e99e337e39c8519, State: Initialized, Role: FOLLOWER
I20260807 19:48:19.608840 18447 ts_tablet_manager.cc:1428] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 19:48:19.608836 18446 consensus_queue.cc:260] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.609987 18424 heartbeater.cc:503] Master 127.0.198.254:33813 was elected leader, sending a full tablet report...
I20260807 19:48:19.610646 18446 ts_tablet_manager.cc:1428] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260807 19:48:19.616034 18448 raft_consensus.cc:348] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.616470 18448 raft_consensus.cc:374] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:19.616577 18448 raft_consensus.cc:729] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c8d439d041482e9f67c984cb375772, State: Initialized, Role: FOLLOWER
I20260807 19:48:19.617250 18448 consensus_queue.cc:260] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.618903 18448 ts_tablet_manager.cc:1428] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
W20260807 19:48:19.659066 18280 tablet.cc:2369] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:48:19.663589 18452 raft_consensus.cc:482] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:19.664059 18452 raft_consensus.cc:504] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.666029 18452 leader_election.cc:288] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001), b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167)
I20260807 19:48:19.673004 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.666602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:35593 (local address 127.0.198.195:42001)
0807 19:48:19.666828 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.666833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.667252 (+ 419us) server_negotiation.cc:407] Connection header received
0807 19:48:19.667555 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.667561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.667646 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.667714 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:48:19.668420 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.669226 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.671069 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.671997 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.672198 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.672293 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.672590 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.672735 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:48:19.673102 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.666390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:42001 (local address 127.0.198.194:35593)
0807 19:48:19.667099 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.667120 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.667292 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.667926 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.667929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.667951 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:48:19.668268 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.668276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.669381 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.669386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.670898 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.670911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.672237 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.672243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.672415 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.672440 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.672549 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.672698 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":350,"threads_started":1}
I20260807 19:48:19.673727 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.667608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:40873 (local address 127.0.198.193:35167)
0807 19:48:19.667889 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.667896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.667981 (+ 85us) server_negotiation.cc:407] Connection header received
0807 19:48:19.668255 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.668260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.668333 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.668437 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:48:19.669013 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.669745 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.671647 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.672573 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.672754 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.672859 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.673304 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.673446 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:48:19.674429 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.667185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.198.194:40873)
0807 19:48:19.667860 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.667880 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:48:19.668082 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:48:19.668462 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:48:19.668466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:48:19.668491 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:48:19.668840 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.668848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.669911 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.669916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.671481 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:48:19.671494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.672856 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.672862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:48:19.673070 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.673115 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:48:19.673926 (+ 811us) client_negotiation.cc:240] Negotiation successful
0807 19:48:19.674107 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":341,"threads_started":1}
I20260807 19:48:19.674536 18375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93698ac442e442bae0e5fafd444e084" candidate_uuid: "3170b2ff799642299003ada31ba6862c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8f1af05c5d48208e99e337e39c8519" is_pre_election: true
I20260807 19:48:19.675154 18375 raft_consensus.cc:2454] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3170b2ff799642299003ada31ba6862c in term 0.
I20260807 19:48:19.675612 18083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93698ac442e442bae0e5fafd444e084" candidate_uuid: "3170b2ff799642299003ada31ba6862c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0c8d439d041482e9f67c984cb375772" is_pre_election: true
I20260807 19:48:19.675999 18158 leader_election.cc:302] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3170b2ff799642299003ada31ba6862c, 3b8f1af05c5d48208e99e337e39c8519; no voters:
I20260807 19:48:19.676142 18083 raft_consensus.cc:2454] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3170b2ff799642299003ada31ba6862c in term 0.
I20260807 19:48:19.676476 18452 raft_consensus.cc:2781] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:19.676642 18452 raft_consensus.cc:482] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:19.676761 18452 raft_consensus.cc:3037] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:19.679185 18452 raft_consensus.cc:504] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.679939 18452 leader_election.cc:288] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [CANDIDATE]: Term 1 election: Requested vote from peers 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001), b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167)
I20260807 19:48:19.680590 18375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93698ac442e442bae0e5fafd444e084" candidate_uuid: "3170b2ff799642299003ada31ba6862c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8f1af05c5d48208e99e337e39c8519"
I20260807 19:48:19.680675 18083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93698ac442e442bae0e5fafd444e084" candidate_uuid: "3170b2ff799642299003ada31ba6862c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0c8d439d041482e9f67c984cb375772"
I20260807 19:48:19.680836 18375 raft_consensus.cc:3037] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:19.680981 18083 raft_consensus.cc:3037] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:19.683449 18375 raft_consensus.cc:2454] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3170b2ff799642299003ada31ba6862c in term 1.
I20260807 19:48:19.683485 18083 raft_consensus.cc:2454] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3170b2ff799642299003ada31ba6862c in term 1.
I20260807 19:48:19.684048 18158 leader_election.cc:302] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3170b2ff799642299003ada31ba6862c, b0c8d439d041482e9f67c984cb375772; no voters:
I20260807 19:48:19.684492 18452 raft_consensus.cc:2781] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:19.685005 18452 raft_consensus.cc:686] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 1 LEADER]: Becoming Leader. State: Replica: 3170b2ff799642299003ada31ba6862c, State: Running, Role: LEADER
I20260807 19:48:19.685547 18452 consensus_queue.cc:237] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } }
I20260807 19:48:19.692711 17935 catalog_manager.cc:5495] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3170b2ff799642299003ada31ba6862c (127.0.198.194). New cstate: current_term: 1 leader_uuid: "3170b2ff799642299003ada31ba6862c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3170b2ff799642299003ada31ba6862c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 42353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 } health_report { overall_health: UNKNOWN } } }
W20260807 19:48:19.699754 18425 tablet.cc:2369] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:48:19.720026 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.713506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.0.198.194:42353)
0807 19:48:19.713695 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.713701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.713821 (+ 120us) server_negotiation.cc:407] Connection header received
0807 19:48:19.714081 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.714085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.714206 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.714407 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:48:19.714935 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.715770 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.717455 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.717903 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.718020 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.718678 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:48:19.719271 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:48:19.719524 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.719704 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.719786 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:48:19.733621 18083 raft_consensus.cc:1261] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Refusing update from remote peer 3170b2ff799642299003ada31ba6862c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:19.733621 18375 raft_consensus.cc:1261] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Refusing update from remote peer 3170b2ff799642299003ada31ba6862c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:19.734978 18457 consensus_queue.cc:1035] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0c8d439d041482e9f67c984cb375772" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:19.735487 18452 consensus_queue.cc:1035] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8f1af05c5d48208e99e337e39c8519" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 42001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:19.750459 18460 mvcc.cc:205] Tried to move back new op lower bound from 7315997080489852928 to 7315997080316145664. Current Snapshot: MvccSnapshot[applied={T|T < 7315997080489852928}]
I20260807 19:48:19.770778 17937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:48:19.771090 17937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:48:19.774967 17937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Sending DeleteTablet for 3 replicas of tablet e93698ac442e442bae0e5fafd444e084
I20260807 19:48:19.776404 18208 tablet_service.cc:1514] Processing DeleteTablet for tablet e93698ac442e442bae0e5fafd444e084 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:48:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:47336
I20260807 19:48:19.776935 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17974
I20260807 19:48:19.776886 18353 tablet_service.cc:1514] Processing DeleteTablet for tablet e93698ac442e442bae0e5fafd444e084 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:48:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:44024
I20260807 19:48:19.777689 18062 tablet_service.cc:1514] Processing DeleteTablet for tablet e93698ac442e442bae0e5fafd444e084 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:48:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:58410
I20260807 19:48:19.783337 18472 tablet_replica.cc:330] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: stopping tablet replica
I20260807 19:48:19.784153 18472 raft_consensus.cc:2229] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:19.788158 18472 raft_consensus.cc:2258] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:19.796113 18473 tablet_replica.cc:330] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: stopping tablet replica
I20260807 19:48:19.797070 18473 raft_consensus.cc:2229] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:19.797665 18472 ts_tablet_manager.cc:1905] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:48:19.797775 18473 raft_consensus.cc:2258] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:19.800304 18473 ts_tablet_manager.cc:1905] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:48:19.803771 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.779742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.0.198.193:35167)
0807 19:48:19.780005 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.780015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.780042 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:48:19.780118 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.780123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.780217 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.780373 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:19.783043 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.784692 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.787924 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.788969 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.789185 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.794146 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:19.794170 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:19.794190 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:19.794243 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:19.798828 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.800079 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.800088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.800092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.800279 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.800843 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.800848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.800852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.801308 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:19.801643 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.803025 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.803269 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:48:19.804626 18126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:19.808777 18472 ts_tablet_manager.cc:1918] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:48:19.809135 18472 log.cc:1196] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/e93698ac442e442bae0e5fafd444e084
I20260807 19:48:19.809981 18472 ts_tablet_manager.cc:1939] T e93698ac442e442bae0e5fafd444e084 P 3b8f1af05c5d48208e99e337e39c8519: Deleting consensus metadata
I20260807 19:48:19.811722 18471 tablet_replica.cc:330] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: stopping tablet replica
I20260807 19:48:19.812266 18471 raft_consensus.cc:2229] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:19.812773 17921 catalog_manager.cc:4841] TS 3b8f1af05c5d48208e99e337e39c8519 (127.0.198.195:42001): tablet e93698ac442e442bae0e5fafd444e084 (table test-table [id=721e046404c74f8e9cb2a516328e2983]) successfully deleted
I20260807 19:48:19.812963 18471 raft_consensus.cc:2258] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:19.815222 18471 ts_tablet_manager.cc:1905] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:48:19.821890 18471 ts_tablet_manager.cc:1918] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:48:19.822153 18471 log.cc:1196] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/e93698ac442e442bae0e5fafd444e084
I20260807 19:48:19.822926 18471 ts_tablet_manager.cc:1939] T e93698ac442e442bae0e5fafd444e084 P 3170b2ff799642299003ada31ba6862c: Deleting consensus metadata
I20260807 19:48:19.825318 17924 catalog_manager.cc:4841] TS 3170b2ff799642299003ada31ba6862c (127.0.198.194:42353): tablet e93698ac442e442bae0e5fafd444e084 (table test-table [id=721e046404c74f8e9cb2a516328e2983]) successfully deleted
I20260807 19:48:19.920368 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17974
I20260807 19:48:19.920528 18473 ts_tablet_manager.cc:1918] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:48:19.920734 18473 log.cc:1196] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/e93698ac442e442bae0e5fafd444e084
W20260807 19:48:19.943943 17921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:35167 (error 108)
I20260807 19:48:19.944315 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 18137
I20260807 19:48:19.946314 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.945769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58438)
0807 19:48:19.945970 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:19.946118 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 19:48:19.946734 17921 catalog_manager.cc:4584] TS b0c8d439d041482e9f67c984cb375772 (127.0.198.193:35167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e93698ac442e442bae0e5fafd444e084: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
I20260807 19:48:19.963047 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:19.946219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.0.198.194:42353)
0807 19:48:19.946430 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:19.946437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:19.946458 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:48:19.946540 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:19.946547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:19.946641 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:19.946759 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:19.948660 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.949998 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.951579 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:19.952065 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:19.952260 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:19.955917 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:19.955939 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:19.955955 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:19.956000 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:19.959634 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.960569 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.960578 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.960580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.960779 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:19.961229 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:19.961232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:19.961234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:19.961641 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:19.962043 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:19.962383 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 19:48:19.962640 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:48:19.963924 18274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:20.000555 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.000029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58454)
0807 19:48:20.000284 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.000407 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 19:48:20.001753 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.001378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58456)
0807 19:48:20.001532 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.001622 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:48:20.062776 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.062164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58464)
0807 19:48:20.062410 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.062531 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 19:48:20.064182 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.063715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58474)
0807 19:48:20.063893 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.063978 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:48:20.086040 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 18137
I20260807 19:48:20.109527 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 18283
I20260807 19:48:20.125330 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.110342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44034 (local address 127.0.198.195:42001)
0807 19:48:20.110516 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:20.110521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:20.110539 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:48:20.110720 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:20.110724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:20.110797 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:20.110883 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:20.112572 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:20.113385 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:20.114921 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:20.115327 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:20.115527 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:20.118473 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:20.118488 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:20.118505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:20.118546 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:20.121690 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:20.122740 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:20.122748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:20.122750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:20.122980 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:20.123701 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:20.123711 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:20.123713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:20.124120 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:20.124368 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:20.124794 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 19:48:20.124974 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:48:20.126142 18419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:20.145211 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.144623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58480)
0807 19:48:20.144917 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.145023 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 19:48:20.146493 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.146029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58490)
0807 19:48:20.146257 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.146345 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 19:48:20.220849 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 18283
I20260807 19:48:20.242071 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 17902
I20260807 19:48:20.243571 17967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:20.364703 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:20.363664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35167 (local address 127.0.0.1:58492)
0807 19:48:20.364048 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:20.364206 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260807 19:48:20.364822 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 17902
I20260807 19:48:40.392628 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--webserver_interface=127.0.198.254
--webserver_port=41843
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:48:40.777343 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:40.777858 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:40.778059 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:40.786748 18481 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:48:40.786937 18481 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:48:40.787009 18481 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:48:40.787083 18481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:48:40.787166 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:40.787294 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:40.787405 18481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:48:40.787489 18481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:48:40.798216 18481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=41843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:40.799713 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:40.801059 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:40.814245 18486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:40.814285 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:40.814245 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:40.814926 18481 server_base.cc:1034] running on GCE node
I20260807 19:48:40.816026 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:40.817703 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:40.818997 18481 hybrid_clock.cc:648] HybridClock initialized: now 1786132120818909 us; error 65 us; skew 500 ppm
I20260807 19:48:40.819511 18481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:40 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:40.823827 18481 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:48:40.825892 18481 webserver.cc:460] Webserver started at http://127.0.198.254:41843/ using document root <none> and password file <none>
I20260807 19:48:40.826777 18481 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:40.826941 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:40.833624 18481 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:48:40.837330 18496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:40.839036 18481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 19:48:40.839293 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "621792d66ba24fc6b203d0813e474651"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "2a8424580cbaffeb6ba5a4df483012cb"
server_key_iv: "197445c2981bc275c6917e531d97bf66"
server_key_version: "encryptionkey@0"
I20260807 19:48:40.840517 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:40.880602 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:40.886004 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:40.886351 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:40.908416 18481 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33813
I20260807 19:48:40.908468 18549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33813 every 8 connection(s)
I20260807 19:48:40.910159 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:48:40.910859 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18481
I20260807 19:48:40.912111 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:35167
--local_ip_for_outbound_sockets=127.0.198.193
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=41297
--webserver_interface=127.0.198.193
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:48:40.918087 18550 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:48:40.921993 18550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap starting.
I20260807 19:48:40.945005 18550 log.cc:824] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Log is configured to *not* fsync() on all Append() calls
I20260807 19:48:40.965670 18550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:40.966427 18550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Bootstrap complete.
I20260807 19:48:40.977385 18550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:40.978456 18550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Initialized, Role: FOLLOWER
I20260807 19:48:40.979336 18550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:40.979696 18550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:48:40.979856 18550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:48:40.980118 18550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:40.985073 18550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:40.985734 18550 leader_election.cc:302] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 621792d66ba24fc6b203d0813e474651; no voters:
I20260807 19:48:40.986552 18550 leader_election.cc:288] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:48:40.986730 18554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:40.988119 18554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [term 3 LEADER]: Becoming Leader. State: Replica: 621792d66ba24fc6b203d0813e474651, State: Running, Role: LEADER
I20260807 19:48:40.988910 18554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } }
I20260807 19:48:40.989310 18550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:48:40.991878 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 621792d66ba24fc6b203d0813e474651. Latest consensus state: current_term: 3 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:48:40.992887 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:40.992260 18555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "621792d66ba24fc6b203d0813e474651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "621792d66ba24fc6b203d0813e474651" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33813 } } }
I20260807 19:48:40.993453 18555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:41.004819 18561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:48:41.015584 18561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:48:41.017589 18561 catalog_manager.cc:1259] Loaded cluster ID: 7a17befa71a04ee8a8ced9770a2bd222
I20260807 19:48:41.017836 18561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:48:41.022925 18561 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:48:41.025324 18561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Loaded TSK: 0
I20260807 19:48:41.025661 18561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 621792d66ba24fc6b203d0813e474651: Loaded TSK: 1
F20260807 19:48:41.027169 18561 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f34b4ca99ad google::LogMessage::Fail() at ??:0
@ 0x7f34b4cac88a google::LogMessage::SendToLog() at ??:0
@ 0x7f34b4ca94de google::LogMessage::Flush() at ??:0
@ 0x7f34b4caaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f34b8a0d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f34b89fc73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f34ca6d81c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f34ca6d7783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f34ca7632a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f34ca7630c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f34ca7ad5e9 std::function<>::operator()() at ??:0
@ 0x7f34ca6dbcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f34ca6d9c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f34ca761610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f34ca761439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f34b78f5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f34b78f891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f34b78f8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f34b78d3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f34babad6db start_thread at ??:0
@ 0x7f34b3b4361f clone at ??:0
W20260807 19:48:41.376974 18552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:41.377336 18552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:41.377503 18552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:41.386490 18552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:41.386667 18552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:41.386868 18552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:48:41.397737 18552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:35167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=41297
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:41.399314 18552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:41.401255 18552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:41.415849 18601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:41.417456 18602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:41.418201 18604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:41.419127 18552 server_base.cc:1034] running on GCE node
I20260807 19:48:41.420030 18552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:41.421262 18552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:41.422513 18552 hybrid_clock.cc:648] HybridClock initialized: now 1786132121422407 us; error 94 us; skew 500 ppm
I20260807 19:48:41.422993 18552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:41 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132121, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:41.427083 18552 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:48:41.429013 18552 webserver.cc:460] Webserver started at http://127.0.198.193:41297/ using document root <none> and password file <none>
I20260807 19:48:41.429853 18552 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:41.430015 18552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:41.436386 18552 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 19:48:41.439878 18612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:41.441712 18552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:48:41.441949 18552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "b0c8d439d041482e9f67c984cb375772"
format_stamp: "Formatted at 2026-08-07 19:47:54 on dist-test-slave-btw8"
server_key: "8cb294e21c43c7fe1decc775a522bf2c"
server_key_iv: "5581b1abb28e8867e704963175b864e1"
server_key_version: "encryptionkey@0"
I20260807 19:48:41.443048 18552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:41.480057 18552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:41.485543 18552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:41.485929 18552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:41.487330 18552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:41.489753 18619 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 19:48:41.496263 18619 ts_tablet_manager.cc:1852] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 19:48:41.497488 18619 ts_tablet_manager.cc:1905] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:48:41.506404 18619 ts_tablet_manager.cc:1918] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:48:41.506677 18619 log.cc:1196] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/e93698ac442e442bae0e5fafd444e084
I20260807 19:48:41.507292 18619 ts_tablet_manager.cc:1939] T e93698ac442e442bae0e5fafd444e084 P b0c8d439d041482e9f67c984cb375772: Deleting consensus metadata
I20260807 19:48:41.507838 18552 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 19:48:41.507987 18552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.019s user 0.000s sys 0.001s
I20260807 19:48:41.508157 18552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:41.508249 18552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:41.547183 18552 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:35167
I20260807 19:48:41.547269 18731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:35167 every 8 connection(s)
I20260807 19:48:41.548722 18552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:48:41.553419 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18552
I20260807 19:48:41.554288 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:42353
--local_ip_for_outbound_sockets=127.0.198.194
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=34009
--webserver_interface=127.0.198.194
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:48:41.554908 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:41.552225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:42713)
0807 19:48:41.553325 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:41.553648 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":432,"threads_started":1}
W20260807 19:48:41.555831 18733 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:33813 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:33813: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
I20260807 19:48:41.556924 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:41.556456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:38737)
0807 19:48:41.556663 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:41.556765 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:48:41.558050 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:41.557562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:41871)
0807 19:48:41.557773 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:41.557878 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 19:48:41.558346 18733 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:48:41.929840 18737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:41.930207 18737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:41.930372 18737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:41.938227 18737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:41.938423 18737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:41.938655 18737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:48:41.949860 18737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:42353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=34009
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:41.951258 18737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:41.952972 18737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:41.969463 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:41.969560 18744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:41.969481 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:41.970920 18737 server_base.cc:1034] running on GCE node
I20260807 19:48:41.971911 18737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:41.973202 18737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:41.974517 18737 hybrid_clock.cc:648] HybridClock initialized: now 1786132121974466 us; error 72 us; skew 500 ppm
I20260807 19:48:41.974957 18737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:41 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132121, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:41.982712 18737 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:48:41.984843 18737 webserver.cc:460] Webserver started at http://127.0.198.194:34009/ using document root <none> and password file <none>
I20260807 19:48:41.985734 18737 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:41.985924 18737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:41.993134 18737 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:48:41.996860 18753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:41.998473 18737 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:48:41.998818 18737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "3170b2ff799642299003ada31ba6862c"
format_stamp: "Formatted at 2026-08-07 19:47:55 on dist-test-slave-btw8"
server_key: "4ab51342744d82dda586098b4dbde77c"
server_key_iv: "04606f15796c77177dd2ff5b20d0c502"
server_key_version: "encryptionkey@0"
I20260807 19:48:42.000262 18737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:42.015266 18737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:42.020843 18737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:42.021175 18737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:42.022351 18737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:42.024309 18737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:42.024474 18737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.024618 18737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:42.024710 18737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.065228 18737 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:42353
I20260807 19:48:42.065331 18870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:42353 every 8 connection(s)
I20260807 19:48:42.066946 18737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:48:42.071947 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18737
I20260807 19:48:42.072909 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:42001
--local_ip_for_outbound_sockets=127.0.198.195
--tserver_master_addrs=127.0.198.254:33813
--webserver_port=45823
--webserver_interface=127.0.198.195
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:37309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:48:42.072980 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.070167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:35199)
0807 19:48:42.071024 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.071344 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":350,"threads_started":1}
W20260807 19:48:42.073881 18871 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:33813 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:33813: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
I20260807 19:48:42.075099 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.074525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:38687)
0807 19:48:42.074766 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.074893 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:48:42.076449 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.075905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:45423)
0807 19:48:42.076141 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.076265 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:48:42.076833 18871 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:48:42.466259 18874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:42.466593 18874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:42.466773 18874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:48:42.474452 18874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:48:42.474601 18874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:48:42.474958 18874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:48:42.487330 18874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:37309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:42001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=45823
--tserver_master_addrs=127.0.198.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:48:42.488936 18874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:48:42.490813 18874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:48:42.506863 18879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:42.506791 18882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:42.506810 18880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:42.507632 18874 server_base.cc:1034] running on GCE node
I20260807 19:48:42.508399 18874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:48:42.509621 18874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:48:42.510885 18874 hybrid_clock.cc:648] HybridClock initialized: now 1786132122510839 us; error 41 us; skew 500 ppm
I20260807 19:48:42.511256 18874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:48:42 dist-test-slave-btw8 krb5kdc[17317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:48:42.514874 18874 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:48:42.516701 18874 webserver.cc:460] Webserver started at http://127.0.198.195:45823/ using document root <none> and password file <none>
I20260807 19:48:42.517513 18874 fs_manager.cc:359] Metadata directory not provided
I20260807 19:48:42.517684 18874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:42.524348 18874 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:48:42.527938 18890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.529999 18874 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 19:48:42.530252 18874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "3b8f1af05c5d48208e99e337e39c8519"
format_stamp: "Formatted at 2026-08-07 19:47:56 on dist-test-slave-btw8"
server_key: "c80ee5a867f0b16c82e70ecb8f486b9d"
server_key_iv: "ffed117b3f3d4ba9e85140e145535bb4"
server_key_version: "encryptionkey@0"
I20260807 19:48:42.531461 18874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:42.550920 18874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:48:42.556905 18874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:48:42.557284 18874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:42.558559 18874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:48:42.560338 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.559292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:43313)
0807 19:48:42.559873 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.559998 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":277,"threads_started":1}
I20260807 19:48:42.560956 18874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:42.561089 18874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.561223 18874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:48:42.561318 18874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.598794 18874 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:42001
I20260807 19:48:42.598883 19011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:42001 every 8 connection(s)
I20260807 19:48:42.600379 18874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:48:42.601486 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 18874
I20260807 19:48:42.605981 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.603923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:56577)
0807 19:48:42.604741 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.604994 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":294,"threads_started":1}
W20260807 19:48:42.606765 19012 heartbeater.cc:644] Failed to heartbeat to 127.0.198.254:33813 (0 consecutive failures): Network error: Failed to ping master at 127.0.198.254:33813: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
I20260807 19:48:42.607800 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.607316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:59671)
0807 19:48:42.607552 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.607660 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 19:48:42.608783 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:42.608416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:40743)
0807 19:48:42.608574 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:42.608664 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:48:42.609041 19012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:48:43.078383 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:43.077459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:38213)
0807 19:48:43.077988 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:43.078124 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":252,"threads_started":1}
I20260807 19:48:43.562537 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:43.561643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:42389)
0807 19:48:43.562234 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:43.562338 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":285,"threads_started":1}
I20260807 19:48:43.610757 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:43.609797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:45571)
0807 19:48:43.610349 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:43.610479 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":255,"threads_started":1}
I20260807 19:48:44.081077 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:44.080092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:57835)
0807 19:48:44.080739 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:44.080848 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":337,"threads_started":1}
I20260807 19:48:44.564875 19020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:44.563908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:48567)
0807 19:48:44.564555 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:44.564674 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":344,"threads_started":1}
I20260807 19:48:44.613291 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:44.612341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:57991)
0807 19:48:44.612974 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:44.613095 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":356,"threads_started":1}
I20260807 19:48:45.083479 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:45.082541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:52981)
0807 19:48:45.083078 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:45.083201 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":300,"threads_started":1}
I20260807 19:48:45.567088 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:45.566245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:51613)
0807 19:48:45.566814 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:45.566916 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":281,"threads_started":1}
I20260807 19:48:45.615309 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:45.614398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:59205)
0807 19:48:45.614914 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:45.615003 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":303,"threads_started":1}
I20260807 19:48:46.085795 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:46.084801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:45241)
0807 19:48:46.085431 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:46.085535 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":319,"threads_started":1}
I20260807 19:48:46.569237 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:46.568300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:41371)
0807 19:48:46.568901 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:46.569004 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":318,"threads_started":1}
I20260807 19:48:46.617201 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:46.616362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:33583)
0807 19:48:46.616943 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:46.617026 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":316,"threads_started":1}
I20260807 19:48:47.088176 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:47.087066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:51017)
0807 19:48:47.087802 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:47.087954 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":361,"threads_started":1}
I20260807 19:48:47.571185 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:47.570435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:50499)
0807 19:48:47.570917 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:47.571009 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260807 19:48:47.619200 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:47.618334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:51011)
0807 19:48:47.618888 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:47.618987 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":280,"threads_started":1}
I20260807 19:48:48.090382 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:48.089438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:51429)
0807 19:48:48.090080 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:48.090199 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":346,"threads_started":1}
I20260807 19:48:48.573335 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:48.572321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:43163)
0807 19:48:48.572936 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:48.573037 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":264,"threads_started":1}
I20260807 19:48:48.621275 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:48.620349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:59633)
0807 19:48:48.620970 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:48.621083 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":356,"threads_started":1}
I20260807 19:48:49.092803 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:49.091760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:56091)
0807 19:48:49.092364 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:49.092467 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260807 19:48:49.575133 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:49.574354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:43741)
0807 19:48:49.574864 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:49.574963 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":252,"threads_started":1}
I20260807 19:48:49.623526 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:49.622436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:55173)
0807 19:48:49.623079 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:49.623184 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":330,"threads_started":1}
I20260807 19:48:50.094885 19039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:50.093984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:56509)
0807 19:48:50.094540 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:50.094652 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":313,"threads_started":1}
I20260807 19:48:50.577376 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:50.576515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:39693)
0807 19:48:50.577066 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:50.577193 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":314,"threads_started":1}
I20260807 19:48:50.625766 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:50.624730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:52665)
0807 19:48:50.625353 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:50.625472 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":310,"threads_started":1}
I20260807 19:48:51.096894 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:51.096002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:55637)
0807 19:48:51.096571 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:51.096670 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":255,"threads_started":1}
I20260807 19:48:51.579562 19043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:51.578575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:57735)
0807 19:48:51.579192 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:51.579306 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":355,"threads_started":1}
I20260807 19:48:51.627856 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:51.626822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:50555)
0807 19:48:51.627522 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:51.627649 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":341,"threads_started":1}
I20260807 19:48:52.098866 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:52.098067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:53941)
0807 19:48:52.098584 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:52.098687 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":279,"threads_started":1}
I20260807 19:48:52.581619 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:52.580699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:52967)
0807 19:48:52.581280 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:52.581380 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
I20260807 19:48:52.630056 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:52.629186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:60477)
0807 19:48:52.629781 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:52.629878 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":360,"threads_started":1}
I20260807 19:48:53.101069 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:53.100187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:54027)
0807 19:48:53.100753 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:53.100867 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":296,"threads_started":1}
I20260807 19:48:53.583813 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:53.582903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:56053)
0807 19:48:53.583478 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:53.583575 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":281,"threads_started":1}
I20260807 19:48:53.632325 19050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:53.631326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:48621)
0807 19:48:53.631991 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:53.632104 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":337,"threads_started":1}
I20260807 19:48:54.103279 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:54.102378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:42101)
0807 19:48:54.102990 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:54.103093 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":343,"threads_started":1}
I20260807 19:48:54.585852 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:54.584787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:41689)
0807 19:48:54.585482 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:54.585625 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":370,"threads_started":1}
I20260807 19:48:54.634471 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:54.633635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:50193)
0807 19:48:54.634207 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:54.634303 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":288,"threads_started":1}
I20260807 19:48:55.105585 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:55.104534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:40247)
0807 19:48:55.105097 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:55.105219 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":300,"threads_started":1}
I20260807 19:48:55.587991 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:55.587010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:40525)
0807 19:48:55.587640 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:55.587795 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":270,"threads_started":1}
I20260807 19:48:55.636778 19057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:55.635738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:35325)
0807 19:48:55.636442 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:55.636558 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":412,"threads_started":1}
I20260807 19:48:56.107544 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:56.106676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:60945)
0807 19:48:56.107173 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:56.107270 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
I20260807 19:48:56.590440 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:56.589404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:39359)
0807 19:48:56.589999 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:56.590183 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":310,"threads_started":1}
I20260807 19:48:56.638931 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:56.637970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:34535)
0807 19:48:56.638591 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:56.638706 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":312,"threads_started":1}
I20260807 19:48:57.109810 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.108871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.194:60345)
0807 19:48:57.109448 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:57.109577 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":304,"threads_started":1}
I20260807 19:48:57.592939 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.591924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.193:40155)
0807 19:48:57.592615 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:57.592771 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":328,"threads_started":1}
I20260807 19:48:57.608215 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 18552
I20260807 19:48:57.634981 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.609689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.0.198.193:35167)
0807 19:48:57.610528 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:57.610549 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:57.610603 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:48:57.612946 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:57.612957 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:57.613433 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:57.613763 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:57.615410 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.616959 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.618491 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.619267 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.619529 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:57.627644 (+ 8115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:57.627658 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:57.627678 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:57.627748 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:57.631104 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.632221 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.632230 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.632238 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.632450 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.632927 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.632931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.632933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.633577 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:57.633846 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:57.634293 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 19:48:57.634656 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":415,"threads_started":1}
I20260807 19:48:57.636679 18727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:57.641495 19066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.640449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33813 (local address 127.0.198.195:33665)
0807 19:48:57.641075 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 19:48:57.641256 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.254:33813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":310,"threads_started":1}
I20260807 19:48:57.725991 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 18552
I20260807 19:48:57.748831 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 18737
I20260807 19:48:57.770463 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.749942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.0.198.194:42353)
0807 19:48:57.750570 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:57.750585 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:57.750625 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:48:57.752548 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:57.752562 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:57.753050 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:57.753388 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:57.755018 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.756563 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.757973 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.758766 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.758926 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:57.762189 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:57.762199 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:57.762217 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:57.762273 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:57.765788 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.766729 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.766746 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.766755 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.766995 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.767808 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.767812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.767815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.768678 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:57.768975 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:57.769556 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 19:48:57.770148 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":322,"threads_started":1}
I20260807 19:48:57.772298 18866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:57.863401 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 18737
I20260807 19:48:57.884850 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 18874
I20260807 19:48:57.908646 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:48:57.886121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.0.198.195:42001)
0807 19:48:57.886886 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 19:48:57.886902 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:48:57.886945 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:48:57.888720 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:48:57.888731 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:48:57.889193 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:48:57.889575 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:48:57.891160 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.892616 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.894114 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:48:57.894855 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:48:57.895088 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:48:57.898027 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:48:57.898037 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:48:57.898054 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:48:57.898110 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:48:57.903424 (+ 5314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.904772 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.904785 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.904796 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.905105 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:48:57.905696 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:48:57.905702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:48:57.905705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:48:57.906705 (+ 1000us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:48:57.907094 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0807 19:48:57.907724 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 19:48:57.908294 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":359,"threads_started":1}
I20260807 19:48:57.910254 19006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:48:57.998329 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 18874
2026-08-07T19:48:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65705 ms)
[----------] 30 tests from SecurityITest (326881 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:48:58 dist-test-slave-btw8 krb5kdc[19073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:48:58 dist-test-slave-btw8 krb5kdc[19073](info): set up 2 sockets
Aug 07 19:48:58 dist-test-slave-btw8 krb5kdc[19073](info): commencing operation
krb5kdc: starting...
W20260807 19:48:59.231336 795 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:48:59 dist-test-slave-btw8 krb5kdc[19073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:48:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:48:59.608688 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:42261
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:42261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:48:59.993032 19087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:48:59.993460 19087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:48:59.993628 19087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:00.002265 19087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:00.002496 19087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:00.002617 19087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:00.002729 19087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:00.002807 19087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:00.014113 19087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:42261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:42261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:00.015789 19087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:00.017273 19087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:00.031875 19092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:00.031841 19095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:00.031800 19093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:00.032375 19087 server_base.cc:1034] running on GCE node
I20260807 19:49:00.033883 19087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:00.035632 19087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:00.036940 19087 hybrid_clock.cc:648] HybridClock initialized: now 1786132140036880 us; error 55 us; skew 500 ppm
I20260807 19:49:00.037395 19087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:00 dist-test-slave-btw8 krb5kdc[19073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:00.041062 19087 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:00.042903 19087 webserver.cc:460] Webserver started at http://127.0.198.254:43173/ using document root <none> and password file <none>
I20260807 19:49:00.043812 19087 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:00.043982 19087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:00.044339 19087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:00.047762 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "0aa553a8755745e2b62804f37393fdd0"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "8ade059314d7d6677e89d9f584e9ed21"
server_key_iv: "ac687e3e2c7f6d06db622ddc7c5fb9fc"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.048617 19087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "0aa553a8755745e2b62804f37393fdd0"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "8ade059314d7d6677e89d9f584e9ed21"
server_key_iv: "ac687e3e2c7f6d06db622ddc7c5fb9fc"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.054840 19087 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:49:00.059557 19102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:00.061421 19087 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:00.061700 19087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "0aa553a8755745e2b62804f37393fdd0"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "8ade059314d7d6677e89d9f584e9ed21"
server_key_iv: "ac687e3e2c7f6d06db622ddc7c5fb9fc"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.061967 19087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:00.077457 19087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:00.082970 19087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:00.083334 19087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:00.105064 19087 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:42261
I20260807 19:49:00.105073 19157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:42261 every 8 connection(s)
I20260807 19:49:00.106679 19087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:00.110872 19158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:00.115908 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19087
I20260807 19:49:00.116159 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:00.117063 795 external_mini_cluster.cc:1428] Setting key a0f42fb93efdfc4d54a3f3dfaec3c70b
I20260807 19:49:00.124125 19158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: Bootstrap starting.
I20260807 19:49:00.127537 19158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:00.128684 19158 log.cc:824] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:00.131929 19158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: No bootstrap required, opened a new log
Aug 07 19:49:00 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:00.140950 19158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } }
I20260807 19:49:00.141440 19158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:00.141564 19158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aa553a8755745e2b62804f37393fdd0, State: Initialized, Role: FOLLOWER
I20260807 19:49:00.142323 19158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } }
I20260807 19:49:00.142622 19158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:00.142787 19158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:00.142985 19158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:00.143160 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:00.119625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.0.198.254:42261)
0807 19:49:00.120594 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:00.120608 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:00.120668 (+ 60us) server_negotiation.cc:407] Connection header received
0807 19:49:00.122647 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:00.122674 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:00.123126 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:00.123615 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:00.125277 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.126657 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.128243 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.129030 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.129251 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:00.133846 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:00.133857 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:00.133877 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:00.133935 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:00.137827 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:00.138841 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:00.138850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:00.138858 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:00.139085 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:00.139633 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:00.139637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:00.139639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:00.140357 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:00.140714 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:00.141329 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 19:49:00.141867 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":457,"threads_started":1}
I20260807 19:49:00.146093 19158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } }
I20260807 19:49:00.146775 19158 leader_election.cc:302] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aa553a8755745e2b62804f37393fdd0; no voters:
I20260807 19:49:00.147579 19158 leader_election.cc:288] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:00.147814 19163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:00.148617 19163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [term 1 LEADER]: Becoming Leader. State: Replica: 0aa553a8755745e2b62804f37393fdd0, State: Running, Role: LEADER
I20260807 19:49:00.149204 19163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } }
I20260807 19:49:00.149812 19158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:00.152602 19164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0aa553a8755745e2b62804f37393fdd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } } }
I20260807 19:49:00.153162 19164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:00.153420 19165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0aa553a8755745e2b62804f37393fdd0. Latest consensus state: current_term: 1 leader_uuid: "0aa553a8755745e2b62804f37393fdd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aa553a8755745e2b62804f37393fdd0" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 42261 } } }
I20260807 19:49:00.153797 19165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:00.158927 19168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:00.165027 19168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:00.176287 19168 catalog_manager.cc:1347] Generated new cluster ID: 797af27d4df446d3a8bb2370bdb43963
I20260807 19:49:00.176445 19168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:00.218097 19168 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:00.220149 19168 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:00.232354 19168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: Generated new TSK 0
I20260807 19:49:00.233770 19168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:49:00.328917 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:00.707098 19184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:00.707453 19184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:00.707553 19184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:00.714982 19184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:00.715113 19184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:00.715235 19184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:00.726711 19184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:00.728163 19184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:00.729482 19184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:00.744484 19191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:00.744746 19190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:00.744495 19193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:00.745939 19184 server_base.cc:1034] running on GCE node
I20260807 19:49:00.746795 19184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:00.748301 19184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:00.749675 19184 hybrid_clock.cc:648] HybridClock initialized: now 1786132140749602 us; error 58 us; skew 500 ppm
I20260807 19:49:00.750120 19184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:00 dist-test-slave-btw8 krb5kdc[19073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:00.754592 19184 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:49:00.756696 19184 webserver.cc:460] Webserver started at http://127.0.198.193:46249/ using document root <none> and password file <none>
I20260807 19:49:00.757567 19184 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:00.757699 19184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:00.758127 19184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:00.761656 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "29c1ebc0d79549939af5c29dd9f70e4b"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "b5640f5c67371e8d4d5ea2313d3bb7cc"
server_key_iv: "f5936bfd9af152834807a6e22208a944"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.762610 19184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "29c1ebc0d79549939af5c29dd9f70e4b"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "b5640f5c67371e8d4d5ea2313d3bb7cc"
server_key_iv: "f5936bfd9af152834807a6e22208a944"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.768980 19184 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260807 19:49:00.773144 19200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:00.774936 19184 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:49:00.775182 19184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "29c1ebc0d79549939af5c29dd9f70e4b"
format_stamp: "Formatted at 2026-08-07 19:49:00 on dist-test-slave-btw8"
server_key: "b5640f5c67371e8d4d5ea2313d3bb7cc"
server_key_iv: "f5936bfd9af152834807a6e22208a944"
server_key_version: "encryptionkey@0"
I20260807 19:49:00.775529 19184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:00.800823 19184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:00.805456 19184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:00.805788 19184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:00.806895 19184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:00.808382 19184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:00.808460 19184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:00.808609 19184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:00.808671 19184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:00.846509 19184 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:37679
I20260807 19:49:00.846649 19316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:37679 every 8 connection(s)
I20260807 19:49:00.847946 19184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:00.849165 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19184
I20260807 19:49:00.849432 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:00.850445 795 external_mini_cluster.cc:1428] Setting key 9f4e25764d1d34a76774881b17119de6
Aug 07 19:49:00 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:00.876284 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:00.851693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:36663 (local address 127.0.198.254:42261)
0807 19:49:00.852279 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:00.852287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:00.853542 (+ 1255us) server_negotiation.cc:407] Connection header received
0807 19:49:00.857250 (+ 3708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:00.857255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:00.857365 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:00.857477 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:00.860240 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.861489 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.863419 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.864208 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.864387 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:00.869020 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:00.869029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:00.869037 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:00.869092 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:00.871988 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:00.873314 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:00.873319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:00.873321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:00.873471 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:00.874100 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:00.874103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:00.874105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:00.874407 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:00.874549 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:00.875799 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0807 19:49:00.876008 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":280,"threads_started":1}
I20260807 19:49:00.878146 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:00.852377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:42261 (local address 127.0.198.193:36663)
0807 19:49:00.853316 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:00.853386 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:00.856745 (+ 3359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:00.857856 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:00.857874 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:00.858895 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:00.859914 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:00.859957 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.861778 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.861784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:00.863189 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:00.863206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:00.864525 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:00.864532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:00.864902 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:00.865819 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:00.865847 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:00.868725 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:00.872187 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:00.872202 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:00.872228 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:00.873111 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:00.873624 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:00.873633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:00.873640 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:00.873964 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:00.874563 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:00.874584 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:00.875204 (+ 620us) client_negotiation.cc:769] Sending connection context
0807 19:49:00.875674 (+ 470us) client_negotiation.cc:240] Negotiation successful
0807 19:49:00.876364 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":369,"threads_started":1}
I20260807 19:49:00.880082 19317 heartbeater.cc:347] Connected to a master server at 127.0.198.254:42261
I20260807 19:49:00.880482 19317 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:00.881331 19317 heartbeater.cc:511] Master 127.0.198.254:42261 requested a full tablet report, sending...
I20260807 19:49:00.883697 19121 ts_manager.cc:194] Registered new tserver with Master: 29c1ebc0d79549939af5c29dd9f70e4b (127.0.198.193:37679)
I20260807 19:49:00.885226 19121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:36663
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:49:00.936443 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:01.323043 19324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:01.323421 19324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:01.323554 19324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:01.331243 19324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:01.331491 19324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:01.331633 19324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:01.342644 19324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:01.344234 19324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:01.345681 19324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:01.360719 19329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:01.360807 19332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:01.360733 19330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:01.361909 19324 server_base.cc:1034] running on GCE node
I20260807 19:49:01.362583 19324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:01.363941 19324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:01.365196 19324 hybrid_clock.cc:648] HybridClock initialized: now 1786132141365113 us; error 68 us; skew 500 ppm
I20260807 19:49:01.365566 19324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:01 dist-test-slave-btw8 krb5kdc[19073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:01.369798 19324 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:49:01.371587 19324 webserver.cc:460] Webserver started at http://127.0.198.194:35809/ using document root <none> and password file <none>
I20260807 19:49:01.372364 19324 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:01.372479 19324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:01.372871 19324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:01.376362 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "e110e33224704371be252a91f5b5b2dd"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "26f721d28e7a19d58691c5e3a76a6724"
server_key_iv: "0a1d8a91ea27f5703575b699040c2f2a"
server_key_version: "encryptionkey@0"
I20260807 19:49:01.377388 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "e110e33224704371be252a91f5b5b2dd"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "26f721d28e7a19d58691c5e3a76a6724"
server_key_iv: "0a1d8a91ea27f5703575b699040c2f2a"
server_key_version: "encryptionkey@0"
I20260807 19:49:01.384095 19324 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:49:01.388643 19340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:01.390494 19324 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:49:01.390682 19324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "e110e33224704371be252a91f5b5b2dd"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "26f721d28e7a19d58691c5e3a76a6724"
server_key_iv: "0a1d8a91ea27f5703575b699040c2f2a"
server_key_version: "encryptionkey@0"
I20260807 19:49:01.390925 19324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:01.434859 19324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:01.440394 19324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:01.440706 19324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:01.441828 19324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:01.443465 19324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:01.443565 19324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:01.443677 19324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:01.443729 19324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:01.482829 19324 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:34125
I20260807 19:49:01.482882 19458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:34125 every 8 connection(s)
I20260807 19:49:01.484261 19324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:01.485556 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19324
I20260807 19:49:01.485777 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:01.486467 795 external_mini_cluster.cc:1428] Setting key 0cdd0bf8a45033ffacbbefc98d404d0e
Aug 07 19:49:01 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:01.510085 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:01.487690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:36185 (local address 127.0.198.254:42261)
0807 19:49:01.488219 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:01.488225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:01.489797 (+ 1572us) server_negotiation.cc:407] Connection header received
0807 19:49:01.492105 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:01.492114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:01.492224 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:01.492356 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:01.494350 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:01.495643 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:01.497786 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:01.498570 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:01.498750 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:01.502870 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:01.502879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:01.502886 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:01.502936 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:01.506140 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:01.507239 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:01.507243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:01.507246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:01.507473 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:01.508030 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:01.508033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:01.508036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:01.508351 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:01.508476 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:01.509532 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0807 19:49:01.509767 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":289,"threads_started":1}
I20260807 19:49:01.511737 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:01.488514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:42261 (local address 127.0.198.194:36185)
0807 19:49:01.489580 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:01.489634 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:01.491775 (+ 2141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:01.492592 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:01.492602 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:01.493444 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:01.494129 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:01.494149 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:01.495871 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:01.495875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:01.497593 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:01.497608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:01.498874 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:01.498879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:01.499092 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:01.499936 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:01.499962 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:01.502572 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:01.506321 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:01.506334 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:01.506357 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:01.507076 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:01.507601 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:01.507609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:01.507616 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:01.507896 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:01.508518 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:01.508530 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:01.509062 (+ 532us) client_negotiation.cc:769] Sending connection context
0807 19:49:01.509415 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 19:49:01.510044 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":486,"threads_started":1}
I20260807 19:49:01.513490 19459 heartbeater.cc:347] Connected to a master server at 127.0.198.254:42261
I20260807 19:49:01.513880 19459 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:01.514703 19459 heartbeater.cc:511] Master 127.0.198.254:42261 requested a full tablet report, sending...
I20260807 19:49:01.516448 19121 ts_manager.cc:194] Registered new tserver with Master: e110e33224704371be252a91f5b5b2dd (127.0.198.194:34125)
I20260807 19:49:01.517335 19121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:36185
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:49:01.568656 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:49:01.888882 19317 heartbeater.cc:503] Master 127.0.198.254:42261 was elected leader, sending a full tablet report...
W20260807 19:49:01.938702 19465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:01.939046 19465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:01.939149 19465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:01.946918 19465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:01.947046 19465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:01.947160 19465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:01.958041 19465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:42261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:01.959618 19465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:01.961076 19465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:01.976900 19470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:01.976965 19471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:01.976882 19473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:01.981925 19465 server_base.cc:1034] running on GCE node
I20260807 19:49:01.982839 19465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:01.984030 19465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:01.985308 19465 hybrid_clock.cc:648] HybridClock initialized: now 1786132141985263 us; error 47 us; skew 500 ppm
I20260807 19:49:01.985680 19465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:01 dist-test-slave-btw8 krb5kdc[19073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:01.989619 19465 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:49:01.991454 19465 webserver.cc:460] Webserver started at http://127.0.198.195:35261/ using document root <none> and password file <none>
I20260807 19:49:01.992214 19465 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:01.992321 19465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:01.992604 19465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:01.995978 19465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "33dfcba852f94f888bc37d6715382c6d"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "b9ca2350691d1c0855e73425a1e56c86"
server_key_iv: "e71a8aa6ad5ffe28751b1d4b98a06fcf"
server_key_version: "encryptionkey@0"
I20260807 19:49:01.996805 19465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "33dfcba852f94f888bc37d6715382c6d"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "b9ca2350691d1c0855e73425a1e56c86"
server_key_iv: "e71a8aa6ad5ffe28751b1d4b98a06fcf"
server_key_version: "encryptionkey@0"
I20260807 19:49:02.002411 19465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:49:02.006664 19480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:02.008581 19465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:49:02.008798 19465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "33dfcba852f94f888bc37d6715382c6d"
format_stamp: "Formatted at 2026-08-07 19:49:01 on dist-test-slave-btw8"
server_key: "b9ca2350691d1c0855e73425a1e56c86"
server_key_iv: "e71a8aa6ad5ffe28751b1d4b98a06fcf"
server_key_version: "encryptionkey@0"
I20260807 19:49:02.009033 19465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:02.041530 19465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:02.047333 19465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:02.047678 19465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:02.049296 19465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:02.050958 19465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:02.051102 19465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:02.051241 19465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:02.051319 19465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:02.092851 19465 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:34337
I20260807 19:49:02.092891 19598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:34337 every 8 connection(s)
I20260807 19:49:02.094384 19465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:02.098044 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19465
I20260807 19:49:02.098268 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:02.098764 795 external_mini_cluster.cc:1428] Setting key 93e0097a433736227fcd1e0f8bcf46ac
Aug 07 19:49:02 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:02.119701 19601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.097482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:35251 (local address 127.0.198.254:42261)
0807 19:49:02.098058 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.098067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.099271 (+ 1204us) server_negotiation.cc:407] Connection header received
0807 19:49:02.102580 (+ 3309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.102585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.102708 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.102844 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:02.104832 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.105733 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.107544 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.108024 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.108185 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.112218 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:02.112223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:02.112227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:02.112265 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:02.116216 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.117149 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.117153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.117156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.117306 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.117847 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.117851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.117854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.118205 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:02.118335 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.119078 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.119284 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":261,"threads_started":1}
I20260807 19:49:02.120777 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.098134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:42261 (local address 127.0.198.195:35251)
0807 19:49:02.099054 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.099104 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.102228 (+ 3124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.103124 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.103139 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.103888 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:02.104637 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.104661 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.105931 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.105937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.107291 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.107304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.108303 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.108311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.108576 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.109386 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:02.109412 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:02.111924 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:02.116376 (+ 4452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:02.116385 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:02.116401 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:02.116958 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:02.117451 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:02.117455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:02.117457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:02.117685 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:02.118327 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:02.118334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:02.118694 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.118956 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.119524 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":396,"threads_started":1}
I20260807 19:49:02.122428 19599 heartbeater.cc:347] Connected to a master server at 127.0.198.254:42261
I20260807 19:49:02.122841 19599 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:02.123683 19599 heartbeater.cc:511] Master 127.0.198.254:42261 requested a full tablet report, sending...
I20260807 19:49:02.125351 19121 ts_manager.cc:194] Registered new tserver with Master: 33dfcba852f94f888bc37d6715382c6d (127.0.198.195:34337)
I20260807 19:49:02.126430 19121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:35251
I20260807 19:49:02.128891 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:02.148970 19601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.134326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.0.198.254:42261)
0807 19:49:02.134521 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.134526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.134996 (+ 470us) server_negotiation.cc:407] Connection header received
0807 19:49:02.135223 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.135230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.135456 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.135568 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:02.137260 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.138182 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.139747 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.140215 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.140329 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.143069 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:02.143075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:02.143078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:02.143121 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:02.146009 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.146781 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.146785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.146788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.146928 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.147302 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.147305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.147307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.147876 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:02.148082 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.148483 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.148727 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:49:02.153689 19121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:02.160907 19121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:02.188969 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.175741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:34125 (local address 127.0.0.1:40132)
0807 19:49:02.176384 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.176407 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.176549 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.177780 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.177787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.177824 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:02.178421 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.178436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.181716 (+ 3280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.181725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.184846 (+ 3121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.184868 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.187547 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.187558 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.187949 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.188038 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.188193 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.188458 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":1,"thread_start_us":304,"threads_started":1}
I20260807 19:49:02.189303 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.174521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.0.198.193:37679)
0807 19:49:02.175225 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.175236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.175283 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:49:02.175599 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.175611 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.175902 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.176147 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:02.177300 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.179797 (+ 2497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.184165 (+ 4368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.186610 (+ 2445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.186837 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.187213 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.188706 (+ 1493us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.188939 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":325,"threads_started":1}
I20260807 19:49:02.189328 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.175993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.0.198.194:34125)
0807 19:49:02.177076 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.177084 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.177112 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:49:02.177216 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.177225 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.177487 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.178699 (+ 1212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:02.179062 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.181508 (+ 2446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.185135 (+ 3627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.187177 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.188490 (+ 1313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.188728 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.188817 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.188990 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":833,"threads_started":1}
I20260807 19:49:02.190325 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.174366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:37679 (local address 127.0.0.1:51634)
0807 19:49:02.175127 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.175169 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.175319 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.176542 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.176554 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.176597 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:02.177028 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.177047 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.180027 (+ 2980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.180037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.183873 (+ 3836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.183900 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.187484 (+ 3584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.187496 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.188441 (+ 945us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.188484 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.189602 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.189790 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":544,"thread_start_us":333,"threads_started":1}
I20260807 19:49:02.194235 19250 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 (DEFAULT_TABLE table=test-table [id=76cdc66bccfd405ebdeeedeca1525677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:02.194252 19390 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 (DEFAULT_TABLE table=test-table [id=76cdc66bccfd405ebdeeedeca1525677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:02.195863 19250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd0d9a08174ac6ab9b8179c3504441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.195858 19390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd0d9a08174ac6ab9b8179c3504441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.198397 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.177153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:34337 (local address 127.0.0.1:34910)
0807 19:49:02.188619 (+ 11466us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.188639 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.189701 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.190811 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.190815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.190837 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:02.191068 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.191076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.193228 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.193234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.195319 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.195339 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.197548 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.197556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.197724 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.197774 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.197876 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.198028 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11324,"thread_start_us":11628,"threads_started":1}
I20260807 19:49:02.199378 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.176877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.0.198.195:34337)
0807 19:49:02.188339 (+ 11462us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.188349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.188746 (+ 397us) server_negotiation.cc:407] Connection header received
0807 19:49:02.189926 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.189938 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.190262 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.190517 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:02.191336 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.193041 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.195570 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.197031 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.197175 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.197402 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.198797 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.198991 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11241,"thread_start_us":11049,"threads_started":1}
I20260807 19:49:02.203784 19530 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 (DEFAULT_TABLE table=test-table [id=76cdc66bccfd405ebdeeedeca1525677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:02.205405 19530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd0d9a08174ac6ab9b8179c3504441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.209944 19620 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Bootstrap starting.
I20260807 19:49:02.212329 19621 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Bootstrap starting.
I20260807 19:49:02.215487 19620 tablet_bootstrap.cc:654] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:02.216331 19621 tablet_bootstrap.cc:654] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:02.217204 19620 log.cc:824] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:02.217708 19621 log.cc:824] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:02.221041 19620 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: No bootstrap required, opened a new log
I20260807 19:49:02.221408 19620 ts_tablet_manager.cc:1397] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260807 19:49:02.221789 19621 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: No bootstrap required, opened a new log
I20260807 19:49:02.222165 19621 ts_tablet_manager.cc:1397] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 19:49:02.222646 19624 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Bootstrap starting.
I20260807 19:49:02.227459 19624 tablet_bootstrap.cc:654] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:02.229065 19624 log.cc:824] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:02.230208 19621 raft_consensus.cc:348] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.230867 19621 raft_consensus.cc:374] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:02.230969 19621 raft_consensus.cc:729] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e110e33224704371be252a91f5b5b2dd, State: Initialized, Role: FOLLOWER
I20260807 19:49:02.231732 19621 consensus_queue.cc:260] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.233243 19459 heartbeater.cc:503] Master 127.0.198.254:42261 was elected leader, sending a full tablet report...
I20260807 19:49:02.234062 19621 ts_tablet_manager.cc:1428] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260807 19:49:02.234813 19620 raft_consensus.cc:348] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.235487 19620 raft_consensus.cc:374] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:02.235553 19624 tablet_bootstrap.cc:492] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: No bootstrap required, opened a new log
I20260807 19:49:02.235632 19620 raft_consensus.cc:729] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c1ebc0d79549939af5c29dd9f70e4b, State: Initialized, Role: FOLLOWER
I20260807 19:49:02.235899 19624 ts_tablet_manager.cc:1397] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
W20260807 19:49:02.236477 19460 tablet.cc:2369] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:02.236552 19620 consensus_queue.cc:260] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.240784 19620 ts_tablet_manager.cc:1428] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Time spent starting tablet: real 0.019s user 0.019s sys 0.001s
I20260807 19:49:02.245244 19624 raft_consensus.cc:348] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.245698 19624 raft_consensus.cc:374] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:02.245808 19624 raft_consensus.cc:729] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33dfcba852f94f888bc37d6715382c6d, State: Initialized, Role: FOLLOWER
I20260807 19:49:02.246558 19624 consensus_queue.cc:260] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.247792 19599 heartbeater.cc:503] Master 127.0.198.254:42261 was elected leader, sending a full tablet report...
I20260807 19:49:02.248310 19624 ts_tablet_manager.cc:1428] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260807 19:49:02.346499 19600 tablet.cc:2369] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:02.351188 19318 tablet.cc:2369] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:02.574033 19628 raft_consensus.cc:482] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:02.574431 19628 raft_consensus.cc:504] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.576323 19628 leader_election.cc:288] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c1ebc0d79549939af5c29dd9f70e4b (127.0.198.193:37679), e110e33224704371be252a91f5b5b2dd (127.0.198.194:34125)
I20260807 19:49:02.582630 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.576766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:37679 (local address 127.0.198.195:60677)
0807 19:49:02.577040 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.577066 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.577235 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.577883 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.577887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.577913 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:02.578071 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.578081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.579129 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.579137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.580683 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.580695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.581873 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.581880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.582065 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.582091 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.582197 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.582339 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 19:49:02.582628 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.576867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:60677 (local address 127.0.198.193:37679)
0807 19:49:02.577137 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.577144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.577167 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:49:02.577461 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.577469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.577574 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.577697 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:02.578233 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.578966 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.580851 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.581703 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.581878 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.581959 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.582238 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.582371 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:49:02.583612 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.577254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:34125 (local address 127.0.198.195:44347)
0807 19:49:02.577910 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:02.577924 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:02.578081 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:02.578427 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:02.578432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:02.578455 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:02.578864 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.578872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.580075 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.580081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.581638 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:02.581651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.582816 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.582823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:02.583013 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.583056 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 19:49:02.583162 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:49:02.583296 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":403,"threads_started":1}
I20260807 19:49:02.583639 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.577567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:44347 (local address 127.0.198.194:34125)
0807 19:49:02.577797 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.577804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.578015 (+ 211us) server_negotiation.cc:407] Connection header received
0807 19:49:02.578228 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.578234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.578304 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.578410 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:02.579035 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.579898 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.581796 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.582600 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.582793 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.582869 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.583211 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.583317 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:49:02.584069 19272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20dd0d9a08174ac6ab9b8179c3504441" candidate_uuid: "33dfcba852f94f888bc37d6715382c6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" is_pre_election: true
I20260807 19:49:02.584910 19272 raft_consensus.cc:2454] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33dfcba852f94f888bc37d6715382c6d in term 0.
I20260807 19:49:02.584849 19412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20dd0d9a08174ac6ab9b8179c3504441" candidate_uuid: "33dfcba852f94f888bc37d6715382c6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e110e33224704371be252a91f5b5b2dd" is_pre_election: true
I20260807 19:49:02.585435 19412 raft_consensus.cc:2454] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33dfcba852f94f888bc37d6715382c6d in term 0.
I20260807 19:49:02.585803 19483 leader_election.cc:302] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c1ebc0d79549939af5c29dd9f70e4b, 33dfcba852f94f888bc37d6715382c6d; no voters:
I20260807 19:49:02.586264 19628 raft_consensus.cc:2781] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:02.586421 19628 raft_consensus.cc:482] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:02.586536 19628 raft_consensus.cc:3037] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:02.588905 19628 raft_consensus.cc:504] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.589566 19628 leader_election.cc:288] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [CANDIDATE]: Term 1 election: Requested vote from peers 29c1ebc0d79549939af5c29dd9f70e4b (127.0.198.193:37679), e110e33224704371be252a91f5b5b2dd (127.0.198.194:34125)
I20260807 19:49:02.590180 19412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20dd0d9a08174ac6ab9b8179c3504441" candidate_uuid: "33dfcba852f94f888bc37d6715382c6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e110e33224704371be252a91f5b5b2dd"
I20260807 19:49:02.590212 19272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20dd0d9a08174ac6ab9b8179c3504441" candidate_uuid: "33dfcba852f94f888bc37d6715382c6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c1ebc0d79549939af5c29dd9f70e4b"
I20260807 19:49:02.590420 19412 raft_consensus.cc:3037] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:02.590447 19272 raft_consensus.cc:3037] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:02.592990 19412 raft_consensus.cc:2454] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33dfcba852f94f888bc37d6715382c6d in term 1.
I20260807 19:49:02.593001 19272 raft_consensus.cc:2454] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33dfcba852f94f888bc37d6715382c6d in term 1.
I20260807 19:49:02.593495 19483 leader_election.cc:302] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33dfcba852f94f888bc37d6715382c6d, e110e33224704371be252a91f5b5b2dd; no voters:
I20260807 19:49:02.593851 19628 raft_consensus.cc:2781] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:02.594389 19628 raft_consensus.cc:686] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 1 LEADER]: Becoming Leader. State: Replica: 33dfcba852f94f888bc37d6715382c6d, State: Running, Role: LEADER
I20260807 19:49:02.595008 19628 consensus_queue.cc:237] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } }
I20260807 19:49:02.600224 19121 catalog_manager.cc:5495] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 33dfcba852f94f888bc37d6715382c6d (127.0.198.195). New cstate: current_term: 1 leader_uuid: "33dfcba852f94f888bc37d6715382c6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33dfcba852f94f888bc37d6715382c6d" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 34337 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:02.664893 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.657851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.0.198.195:34337)
0807 19:49:02.658183 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.658193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.658848 (+ 655us) server_negotiation.cc:407] Connection header received
0807 19:49:02.659142 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.659146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.659235 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.659455 (+ 220us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:02.659944 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.660690 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.662252 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.662693 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.662843 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.663504 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:02.664194 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:02.664356 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.664457 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.664561 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 19:49:02.675155 19272 raft_consensus.cc:1261] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 1 FOLLOWER]: Refusing update from remote peer 33dfcba852f94f888bc37d6715382c6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:02.675155 19412 raft_consensus.cc:1261] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 1 FOLLOWER]: Refusing update from remote peer 33dfcba852f94f888bc37d6715382c6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:02.676578 19628 consensus_queue.cc:1035] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e110e33224704371be252a91f5b5b2dd" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 34125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:02.676982 19630 consensus_queue.cc:1035] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c1ebc0d79549939af5c29dd9f70e4b" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:02.709082 19121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:02.709407 19121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:02.713359 19121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0aa553a8755745e2b62804f37393fdd0: Sending DeleteTablet for 3 replicas of tablet 20dd0d9a08174ac6ab9b8179c3504441
I20260807 19:49:02.714757 19390 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:02 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:40132
I20260807 19:49:02.714715 19250 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:02 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:51634
I20260807 19:49:02.714895 19530 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd0d9a08174ac6ab9b8179c3504441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:02 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:34910
I20260807 19:49:02.715796 19644 tablet_replica.cc:330] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: stopping tablet replica
I20260807 19:49:02.716034 19646 tablet_replica.cc:330] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: stopping tablet replica
I20260807 19:49:02.716331 19644 raft_consensus.cc:2229] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:02.716641 19646 raft_consensus.cc:2229] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:02.716883 19644 raft_consensus.cc:2258] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:02.717006 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19184
I20260807 19:49:02.717562 19646 raft_consensus.cc:2258] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:02.717696 19645 tablet_replica.cc:330] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: stopping tablet replica
I20260807 19:49:02.718115 19645 raft_consensus.cc:2229] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:02.718618 19645 raft_consensus.cc:2258] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:02.721913 19644 ts_tablet_manager.cc:1905] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:02.721976 19645 ts_tablet_manager.cc:1905] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:02.723526 19646 ts_tablet_manager.cc:1905] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:02.729696 19645 ts_tablet_manager.cc:1918] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:02.730007 19645 log.cc:1196] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/20dd0d9a08174ac6ab9b8179c3504441
I20260807 19:49:02.730981 19645 ts_tablet_manager.cc:1939] T 20dd0d9a08174ac6ab9b8179c3504441 P e110e33224704371be252a91f5b5b2dd: Deleting consensus metadata
I20260807 19:49:02.731427 19646 ts_tablet_manager.cc:1918] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:02.731685 19646 log.cc:1196] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/20dd0d9a08174ac6ab9b8179c3504441
I20260807 19:49:02.732348 19646 ts_tablet_manager.cc:1939] T 20dd0d9a08174ac6ab9b8179c3504441 P 33dfcba852f94f888bc37d6715382c6d: Deleting consensus metadata
I20260807 19:49:02.733667 19105 catalog_manager.cc:4841] TS e110e33224704371be252a91f5b5b2dd (127.0.198.194:34125): tablet 20dd0d9a08174ac6ab9b8179c3504441 (table test-table [id=76cdc66bccfd405ebdeeedeca1525677]) successfully deleted
I20260807 19:49:02.734180 19105 catalog_manager.cc:4841] TS 33dfcba852f94f888bc37d6715382c6d (127.0.198.195:34337): tablet 20dd0d9a08174ac6ab9b8179c3504441 (table test-table [id=76cdc66bccfd405ebdeeedeca1525677]) successfully deleted
I20260807 19:49:02.735141 19644 ts_tablet_manager.cc:1918] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:02.735356 19644 log.cc:1196] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/20dd0d9a08174ac6ab9b8179c3504441
I20260807 19:49:02.735917 19644 ts_tablet_manager.cc:1939] T 20dd0d9a08174ac6ab9b8179c3504441 P 29c1ebc0d79549939af5c29dd9f70e4b: Deleting consensus metadata
Aug 07 19:49:02 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:49:02.738884 19105 catalog_manager.cc:4841] TS 29c1ebc0d79549939af5c29dd9f70e4b (127.0.198.193:37679): tablet 20dd0d9a08174ac6ab9b8179c3504441 (table test-table [id=76cdc66bccfd405ebdeeedeca1525677]) successfully deleted
I20260807 19:49:02.748871 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.719043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.0.198.193:37679)
0807 19:49:02.719396 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.719403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.719978 (+ 575us) server_negotiation.cc:407] Connection header received
0807 19:49:02.720340 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.720346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.720430 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.720506 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:02.722513 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.723692 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.725773 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.726438 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.726554 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.740548 (+ 13994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:02.740567 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:02.740583 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:02.740638 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:02.744985 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.746248 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.746257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.746260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.746433 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.746973 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.746978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.746980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.747497 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:02.747791 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.748224 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.748433 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260807 19:49:02.749756 19312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:02.846076 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19184
I20260807 19:49:02.874348 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19324
Aug 07 19:49:02 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:49:02.894073 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:02.875263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.0.198.194:34125)
0807 19:49:02.875471 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:02.875480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:02.875632 (+ 152us) server_negotiation.cc:407] Connection header received
0807 19:49:02.875827 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:02.875831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:02.875900 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:02.875996 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:02.877824 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.878548 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.880642 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:02.881101 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:02.881222 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:02.887229 (+ 6007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:02.887248 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:02.887265 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:02.887314 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:02.890486 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.891527 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.891539 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.891544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.891753 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:02.892182 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:02.892186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:02.892188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:02.892590 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:02.892853 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:02.893393 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 19:49:02.893603 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:49:02.894809 19454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:03.005622 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19324
I20260807 19:49:03.032140 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19465
Aug 07 19:49:03 dist-test-slave-btw8 krb5kdc[19073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:49:03.051084 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:03.033181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.0.198.195:34337)
0807 19:49:03.033392 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:03.033397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:03.033421 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:49:03.033493 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:03.033500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:03.033606 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:03.033741 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:03.035158 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:03.036083 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:03.037589 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:03.038065 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:03.038235 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:03.043639 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:03.043657 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:03.043673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:03.043721 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:03.047230 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:03.048345 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:03.048354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:03.048356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:03.048542 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:03.049080 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:03.049085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:03.049088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:03.049654 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:03.049963 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:03.050443 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:49:03.050714 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:49:03.052048 19593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:03.195575 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19465
I20260807 19:49:03.225310 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19087
I20260807 19:49:03.226809 19152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:03.342068 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19087
2026-08-07T19:49:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5352 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T19:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:03Z Disabled control of system clock
I20260807 19:49:03.454416 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44071
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:44071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:03.825265 19656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:03.825608 19656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:03.825763 19656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:03.833495 19656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:03.833643 19656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:03.833714 19656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:03.833812 19656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:03.833920 19656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:03.843806 19656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:44071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:44071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:03.845342 19656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:03.847404 19656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:03.860745 19662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:03.860745 19661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:03.860707 19664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:03.861882 19656 server_base.cc:1034] running on GCE node
I20260807 19:49:03.862931 19656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:03.864748 19656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:03.866048 19656 hybrid_clock.cc:648] HybridClock initialized: now 1786132143866003 us; error 45 us; skew 500 ppm
I20260807 19:49:03.866523 19656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:03.869513 19656 webserver.cc:460] Webserver started at http://127.0.198.254:33099/ using document root <none> and password file <none>
I20260807 19:49:03.870523 19656 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:03.870675 19656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:03.871150 19656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:03.875051 19656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "d0b369d99dc443cfba195d763f70f89a"
format_stamp: "Formatted at 2026-08-07 19:49:03 on dist-test-slave-btw8"
server_key: "b51cac1a0fcc58060915ab156b70484a"
server_key_iv: "0abd8d1b19183c3f7201e8fdceec2485"
server_key_version: "encryptionkey@0"
I20260807 19:49:03.876055 19656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "d0b369d99dc443cfba195d763f70f89a"
format_stamp: "Formatted at 2026-08-07 19:49:03 on dist-test-slave-btw8"
server_key: "b51cac1a0fcc58060915ab156b70484a"
server_key_iv: "0abd8d1b19183c3f7201e8fdceec2485"
server_key_version: "encryptionkey@0"
I20260807 19:49:03.882128 19656 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 19:49:03.886639 19670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:03.888563 19656 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:03.888827 19656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "d0b369d99dc443cfba195d763f70f89a"
format_stamp: "Formatted at 2026-08-07 19:49:03 on dist-test-slave-btw8"
server_key: "b51cac1a0fcc58060915ab156b70484a"
server_key_iv: "0abd8d1b19183c3f7201e8fdceec2485"
server_key_version: "encryptionkey@0"
I20260807 19:49:03.889097 19656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:03.911314 19656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:03.912417 19656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:03.912773 19656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:03.934724 19656 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:44071
I20260807 19:49:03.934759 19723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:44071 every 8 connection(s)
I20260807 19:49:03.936486 19656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:03.940896 19724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:03.944300 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19656
I20260807 19:49:03.944512 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:03.945091 795 external_mini_cluster.cc:1428] Setting key 9f36863025e6722c233f813f415a6260
I20260807 19:49:03.955871 19724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: Bootstrap starting.
I20260807 19:49:03.960386 19724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:03.961849 19724 log.cc:824] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:03.962600 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:03.947442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50300 (local address 127.0.198.254:44071)
0807 19:49:03.948572 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:03.948589 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:03.948646 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:49:03.950837 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:03.950876 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:03.951623 (+ 747us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:03.952112 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:03.952741 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:03.954765 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:03.956490 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:03.957572 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:03.957815 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:03.959192 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:03.959205 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:03.959231 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:03.960045 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:03.960194 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:03.960235 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:03.960394 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:03.960713 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:49:03.961133 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":377,"threads_started":1}
I20260807 19:49:03.966003 19724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: No bootstrap required, opened a new log
I20260807 19:49:03.974488 19724 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } }
I20260807 19:49:03.975004 19724 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:03.975148 19724 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b369d99dc443cfba195d763f70f89a, State: Initialized, Role: FOLLOWER
I20260807 19:49:03.975895 19724 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } }
I20260807 19:49:03.976184 19724 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:03.976357 19724 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:03.976605 19724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:03.979139 19724 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } }
I20260807 19:49:03.979878 19724 leader_election.cc:302] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b369d99dc443cfba195d763f70f89a; no voters:
I20260807 19:49:03.980607 19724 leader_election.cc:288] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:03.980844 19729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:03.981751 19729 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [term 1 LEADER]: Becoming Leader. State: Replica: d0b369d99dc443cfba195d763f70f89a, State: Running, Role: LEADER
I20260807 19:49:03.982365 19729 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } }
I20260807 19:49:03.983040 19724 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:03.985463 19731 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b369d99dc443cfba195d763f70f89a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } } }
I20260807 19:49:03.985941 19731 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:03.985814 19732 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b369d99dc443cfba195d763f70f89a. Latest consensus state: current_term: 1 leader_uuid: "d0b369d99dc443cfba195d763f70f89a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b369d99dc443cfba195d763f70f89a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 44071 } } }
I20260807 19:49:03.986238 19732 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:03.988356 19736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:03.994203 19736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:04.004637 19736 catalog_manager.cc:1347] Generated new cluster ID: 933fea1e818845e3bdc50535e4aa10bb
I20260807 19:49:04.004783 19736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:04.016129 19736 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:04.018113 19736 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:04.037758 19736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: Generated new TSK 0
I20260807 19:49:04.039004 19736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:04.044199 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--builtin_ntp_servers=127.0.198.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:04.422224 19750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:04.422554 19750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:04.422714 19750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:04.430459 19750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:04.430583 19750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:04.430763 19750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:04.441298 19750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:04.442740 19750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:04.444718 19750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:04.459713 19756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:04.459802 19755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:04.459854 19758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:04.460893 19750 server_base.cc:1034] running on GCE node
I20260807 19:49:04.461639 19750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:04.462809 19750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:04.464074 19750 hybrid_clock.cc:648] HybridClock initialized: now 1786132144463998 us; error 70 us; skew 500 ppm
I20260807 19:49:04.464488 19750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:04.467617 19750 webserver.cc:460] Webserver started at http://127.0.198.193:44449/ using document root <none> and password file <none>
I20260807 19:49:04.468532 19750 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:04.468688 19750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:04.469059 19750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:04.472455 19750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "7bb8532eab24499dacdf2b5f64ef0580"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "60ddb0ad6da65b586677430dae0ba489"
server_key_iv: "5d7701c77a7a6802eb47a52d6945d04b"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.473367 19750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "7bb8532eab24499dacdf2b5f64ef0580"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "60ddb0ad6da65b586677430dae0ba489"
server_key_iv: "5d7701c77a7a6802eb47a52d6945d04b"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.479462 19750 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:49:04.483724 19764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:04.485527 19750 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:49:04.485924 19750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "7bb8532eab24499dacdf2b5f64ef0580"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "60ddb0ad6da65b586677430dae0ba489"
server_key_iv: "5d7701c77a7a6802eb47a52d6945d04b"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.486228 19750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:04.508664 19750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:04.509634 19750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:04.509987 19750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:04.511041 19750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:04.512774 19750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:04.512866 19750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:04.512943 19750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:04.513000 19750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:04.550203 19750 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:43485
I20260807 19:49:04.550307 19880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:43485 every 8 connection(s)
I20260807 19:49:04.551757 19750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:04.552402 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19750
I20260807 19:49:04.552611 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:04.553287 795 external_mini_cluster.cc:1428] Setting key 4af79a87478c71724c5d692784218ea3
I20260807 19:49:04.556645 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--builtin_ntp_servers=127.0.198.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:04.569039 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:04.554812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:43297 (local address 127.0.198.254:44071)
0807 19:49:04.555426 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:04.555432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:04.559047 (+ 3615us) server_negotiation.cc:407] Connection header received
0807 19:49:04.559123 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:04.559127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:04.559202 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:04.559294 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:04.561069 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:04.562195 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:04.565135 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:04.565819 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:04.565963 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:04.567981 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:04.567989 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:04.567997 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:04.568026 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:04.568078 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:04.568102 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:04.568185 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:04.568487 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 19:49:04.568609 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":323,"threads_started":1}
I20260807 19:49:04.570787 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:04.555593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44071 (local address 127.0.198.193:43297)
0807 19:49:04.556792 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:04.556855 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:04.558637 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:04.559696 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:04.559713 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:04.559857 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:04.560809 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:04.560840 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:04.562425 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:04.562433 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:04.564912 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:04.564935 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:04.566116 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:04.566126 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:04.566507 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:04.567512 (+ 1005us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:04.567518 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:04.567582 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:04.567716 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:04.568212 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:04.568232 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:04.568245 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 19:49:04.568445 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 19:49:04.568952 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":373,"threads_started":1}
I20260807 19:49:04.572925 19881 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44071
I20260807 19:49:04.573275 19881 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:04.574187 19881 heartbeater.cc:511] Master 127.0.198.254:44071 requested a full tablet report, sending...
I20260807 19:49:04.576941 19687 ts_manager.cc:194] Registered new tserver with Master: 7bb8532eab24499dacdf2b5f64ef0580 (127.0.198.193:43485)
I20260807 19:49:04.579344 19687 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.193:43297
W20260807 19:49:04.927546 19885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:04.927884 19885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:04.927976 19885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:04.935741 19885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:04.935863 19885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:04.936040 19885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:04.945950 19885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:04.947317 19885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:04.949275 19885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:04.964658 19891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:04.964592 19890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:04.964617 19894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:04.969280 19885 server_base.cc:1034] running on GCE node
I20260807 19:49:04.970124 19885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:04.971258 19885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:04.972693 19885 hybrid_clock.cc:648] HybridClock initialized: now 1786132144972617 us; error 59 us; skew 500 ppm
I20260807 19:49:04.973109 19885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:04.976481 19885 webserver.cc:460] Webserver started at http://127.0.198.194:42311/ using document root <none> and password file <none>
I20260807 19:49:04.977305 19885 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:04.977452 19885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:04.977864 19885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:04.981195 19885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "44b5544790f3471bbde8a710747834fa"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "fefc75f201afa1ed901f71fc2bdfe004"
server_key_iv: "569df599873710d318e09f8347c3441a"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.982108 19885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "44b5544790f3471bbde8a710747834fa"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "fefc75f201afa1ed901f71fc2bdfe004"
server_key_iv: "569df599873710d318e09f8347c3441a"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.988194 19885 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:49:04.992288 19900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:04.993898 19885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:49:04.994083 19885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "44b5544790f3471bbde8a710747834fa"
format_stamp: "Formatted at 2026-08-07 19:49:04 on dist-test-slave-btw8"
server_key: "fefc75f201afa1ed901f71fc2bdfe004"
server_key_iv: "569df599873710d318e09f8347c3441a"
server_key_version: "encryptionkey@0"
I20260807 19:49:04.994330 19885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:05.014992 19885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:05.015892 19885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:05.016222 19885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:05.017336 19885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:05.019001 19885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:05.019091 19885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:05.019213 19885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:05.019284 19885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:05.054774 19885 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35677
I20260807 19:49:05.054908 20017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35677 every 8 connection(s)
I20260807 19:49:05.056335 19885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:05.058187 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 19885
I20260807 19:49:05.058396 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:05.058940 795 external_mini_cluster.cc:1428] Setting key d4d65fd82b858bc7ba355bd601f5ca2e
I20260807 19:49:05.062392 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--builtin_ntp_servers=127.0.198.212:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:05.072422 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.059815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:41241 (local address 127.0.198.254:44071)
0807 19:49:05.060054 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.060061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.062092 (+ 2031us) server_negotiation.cc:407] Connection header received
0807 19:49:05.064285 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.064294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.064398 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.064523 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:05.065711 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.066875 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.068691 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.069425 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.069581 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.071408 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:05.071416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:05.071422 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:05.071455 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:05.071494 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:05.071509 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:05.071632 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.071909 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.072042 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:49:05.073743 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.060583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44071 (local address 127.0.198.194:41241)
0807 19:49:05.061885 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.061944 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.063936 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.064747 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.064756 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.064859 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:05.065510 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.065527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.067085 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.067090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.068529 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.068544 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.069730 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.069737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.069983 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.070899 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:05.070902 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:05.070951 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:05.071054 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:05.071632 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:05.071649 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:05.071661 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.071897 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.072431 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":451,"threads_started":1}
I20260807 19:49:05.075670 20018 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44071
I20260807 19:49:05.076056 20018 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:05.077297 20018 heartbeater.cc:511] Master 127.0.198.254:44071 requested a full tablet report, sending...
I20260807 19:49:05.079404 19687 ts_manager.cc:194] Registered new tserver with Master: 44b5544790f3471bbde8a710747834fa (127.0.198.194:35677)
I20260807 19:49:05.080349 19687 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.194:41241
W20260807 19:49:05.435777 20021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:05.436107 20021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:05.436251 20021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:05.444042 20021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:05.444173 20021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:05.444370 20021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:05.454110 20021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:05.455610 20021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:05.457731 20021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:05.472793 20029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:05.472945 20027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:05.473456 20026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:05.473704 20021 server_base.cc:1034] running on GCE node
I20260807 19:49:05.474448 20021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:05.475728 20021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:05.477054 20021 hybrid_clock.cc:648] HybridClock initialized: now 1786132145476976 us; error 60 us; skew 500 ppm
I20260807 19:49:05.477456 20021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:05.480181 20021 webserver.cc:460] Webserver started at http://127.0.198.195:36919/ using document root <none> and password file <none>
I20260807 19:49:05.480983 20021 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:05.481137 20021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:05.481518 20021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:05.485189 20021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "2632aff2d9514c7db14ac640e1495800"
format_stamp: "Formatted at 2026-08-07 19:49:05 on dist-test-slave-btw8"
server_key: "f959fe7dc5baae548de795c555f6aa49"
server_key_iv: "efe855654850c60b4ca60089415e8852"
server_key_version: "encryptionkey@0"
I20260807 19:49:05.486166 20021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "2632aff2d9514c7db14ac640e1495800"
format_stamp: "Formatted at 2026-08-07 19:49:05 on dist-test-slave-btw8"
server_key: "f959fe7dc5baae548de795c555f6aa49"
server_key_iv: "efe855654850c60b4ca60089415e8852"
server_key_version: "encryptionkey@0"
I20260807 19:49:05.492609 20021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:49:05.497538 20036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:05.499436 20021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 19:49:05.499653 20021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "2632aff2d9514c7db14ac640e1495800"
format_stamp: "Formatted at 2026-08-07 19:49:05 on dist-test-slave-btw8"
server_key: "f959fe7dc5baae548de795c555f6aa49"
server_key_iv: "efe855654850c60b4ca60089415e8852"
server_key_version: "encryptionkey@0"
I20260807 19:49:05.499969 20021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:05.514639 20021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:05.515944 20021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:05.516315 20021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:05.517436 20021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:05.519249 20021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:05.519342 20021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:05.519515 20021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:05.519614 20021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:05.556568 20021 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:46109
I20260807 19:49:05.556700 20155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:46109 every 8 connection(s)
I20260807 19:49:05.558008 20021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:05.563648 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20021
I20260807 19:49:05.563876 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:05.564319 795 external_mini_cluster.cc:1428] Setting key d373d457ef90847ea7cdbfef7fdc8063
I20260807 19:49:05.572775 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.560965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:50893 (local address 127.0.198.254:44071)
0807 19:49:05.561189 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.561194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.562849 (+ 1655us) server_negotiation.cc:407] Connection header received
0807 19:49:05.565301 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.565309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.565407 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.565504 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:05.566978 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.568025 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.569783 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.570183 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.570306 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.571968 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:05.571973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:05.571977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:05.571997 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:05.572030 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:05.572051 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:05.572131 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.572398 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.572489 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:49:05.573967 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.561507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:44071 (local address 127.0.198.195:50893)
0807 19:49:05.562628 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.562688 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.564971 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.565866 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.565876 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.565991 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:05.566754 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.566789 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.568214 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.568218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.569624 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.569636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.570454 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.570469 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.570769 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.571636 (+ 867us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:05.571638 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:05.571682 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:05.571769 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:05.572172 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:05.572182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:05.572188 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.572371 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.572710 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":348,"threads_started":1}
I20260807 19:49:05.575768 20156 heartbeater.cc:347] Connected to a master server at 127.0.198.254:44071
I20260807 19:49:05.576056 20156 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:05.576848 20156 heartbeater.cc:511] Master 127.0.198.254:44071 requested a full tablet report, sending...
I20260807 19:49:05.578455 19687 ts_manager.cc:194] Registered new tserver with Master: 2632aff2d9514c7db14ac640e1495800 (127.0.198.195:46109)
I20260807 19:49:05.579663 19687 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.195:50893
I20260807 19:49:05.580698 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:05.584444 19881 heartbeater.cc:503] Master 127.0.198.254:44071 was elected leader, sending a full tablet report...
I20260807 19:49:05.592916 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.586356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.0.198.254:44071)
0807 19:49:05.586677 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.586682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.587096 (+ 414us) server_negotiation.cc:407] Connection header received
0807 19:49:05.587247 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.587251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.587313 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.587470 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:05.587941 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.588666 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.590368 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.590902 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.591008 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.592140 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:05.592145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:05.592149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:05.592170 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:05.592206 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:05.592239 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:05.592315 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.592442 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.592531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260807 19:49:05.597662 19687 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:05.605187 19687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:05.642637 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.619142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.0.198.193:43485)
0807 19:49:05.630900 (+ 11758us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.630915 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.632077 (+ 1162us) server_negotiation.cc:407] Connection header received
0807 19:49:05.632235 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.632244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.632564 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.633888 (+ 1324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:05.634231 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.636420 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.639819 (+ 3399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.641193 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.641360 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.641678 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.642017 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.642212 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11540,"thread_start_us":11356,"threads_started":1}
I20260807 19:49:05.642851 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.619063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40566)
0807 19:49:05.631537 (+ 12474us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.631586 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.631840 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.632917 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.632930 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.632982 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:05.633579 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.633598 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.636643 (+ 3045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.636652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.639536 (+ 2884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.639553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.641426 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.641432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.641705 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.641778 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.641966 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.642332 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12103,"mutex_wait_us":163,"thread_start_us":11908,"threads_started":1}
I20260807 19:49:05.643087 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.619527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46109 (local address 127.0.0.1:60302)
0807 19:49:05.633034 (+ 13507us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.633055 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.633231 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.633951 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.633955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.633981 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:05.634358 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.634373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.636474 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.636479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.639780 (+ 3301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.639795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.642075 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.642086 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.642392 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.642434 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.642525 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.642682 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13335,"mutex_wait_us":95,"spinlock_wait_cycles":8448,"thread_start_us":13155,"threads_started":1}
I20260807 19:49:05.644008 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.619765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.0.198.195:46109)
0807 19:49:05.632455 (+ 12690us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.632469 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.633478 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 19:49:05.633597 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.633602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.633761 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.633964 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:05.634576 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.636279 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.640038 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.641514 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.641657 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.641935 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.643475 (+ 1540us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.643661 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12426,"thread_start_us":13551,"threads_started":1}
I20260807 19:49:05.646867 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.634416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35677 (local address 127.0.0.1:35708)
0807 19:49:05.637886 (+ 3470us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.637907 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.638034 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.638835 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.638840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.638868 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:05.639332 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.639344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.641446 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.641452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.644266 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.644282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.646004 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.646010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.646185 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.646221 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.646309 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.646459 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3300,"mutex_wait_us":1,"thread_start_us":3069,"threads_started":1}
I20260807 19:49:05.647075 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.632362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.0.198.194:35677)
0807 19:49:05.638136 (+ 5774us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.638148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.638188 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:49:05.638310 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.638320 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.638593 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.638811 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:05.639603 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.641251 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.644536 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.645800 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.645968 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.646216 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.646512 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.646687 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5590,"thread_start_us":5432,"threads_started":1}
I20260807 19:49:05.647459 20086 tablet_service.cc:1467] Processing CreateTablet for tablet 8ada4a978e84422faf25e4689560c167 (DEFAULT_TABLE table=test-table [id=78e9773bb5964c1d8d2753b08d55e75a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:05.647459 19815 tablet_service.cc:1467] Processing CreateTablet for tablet 8ada4a978e84422faf25e4689560c167 (DEFAULT_TABLE table=test-table [id=78e9773bb5964c1d8d2753b08d55e75a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:05.649113 20086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ada4a978e84422faf25e4689560c167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:05.649113 19815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ada4a978e84422faf25e4689560c167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:05.650731 19951 tablet_service.cc:1467] Processing CreateTablet for tablet 8ada4a978e84422faf25e4689560c167 (DEFAULT_TABLE table=test-table [id=78e9773bb5964c1d8d2753b08d55e75a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:05.652047 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ada4a978e84422faf25e4689560c167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:05.666393 20177 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Bootstrap starting.
I20260807 19:49:05.670217 20175 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Bootstrap starting.
I20260807 19:49:05.671520 20177 tablet_bootstrap.cc:654] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:05.672686 20177 log.cc:824] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:05.675549 20175 tablet_bootstrap.cc:654] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:05.676932 20176 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Bootstrap starting.
I20260807 19:49:05.677160 20175 log.cc:824] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:05.681133 20177 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: No bootstrap required, opened a new log
I20260807 19:49:05.681182 20176 tablet_bootstrap.cc:654] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:05.681126 20175 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: No bootstrap required, opened a new log
I20260807 19:49:05.681470 20177 ts_tablet_manager.cc:1397] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260807 19:49:05.681499 20175 ts_tablet_manager.cc:1397] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260807 19:49:05.682552 20176 log.cc:824] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:05.692791 20176 tablet_bootstrap.cc:492] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: No bootstrap required, opened a new log
I20260807 19:49:05.693188 20176 ts_tablet_manager.cc:1397] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.006s
I20260807 19:49:05.692991 20175 raft_consensus.cc:348] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.692991 20177 raft_consensus.cc:348] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.693491 20175 raft_consensus.cc:374] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:05.693497 20177 raft_consensus.cc:374] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:05.693601 20177 raft_consensus.cc:729] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2632aff2d9514c7db14ac640e1495800, State: Initialized, Role: FOLLOWER
I20260807 19:49:05.693600 20175 raft_consensus.cc:729] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b5544790f3471bbde8a710747834fa, State: Initialized, Role: FOLLOWER
I20260807 19:49:05.694350 20177 consensus_queue.cc:260] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.694350 20175 consensus_queue.cc:260] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.695794 20018 heartbeater.cc:503] Master 127.0.198.254:44071 was elected leader, sending a full tablet report...
I20260807 19:49:05.696683 20175 ts_tablet_manager.cc:1428] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 19:49:05.697762 20156 heartbeater.cc:503] Master 127.0.198.254:44071 was elected leader, sending a full tablet report...
I20260807 19:49:05.698383 20177 ts_tablet_manager.cc:1428] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260807 19:49:05.706291 20176 raft_consensus.cc:348] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.706780 20176 raft_consensus.cc:374] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:05.706879 20176 raft_consensus.cc:729] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb8532eab24499dacdf2b5f64ef0580, State: Initialized, Role: FOLLOWER
I20260807 19:49:05.707638 20176 consensus_queue.cc:260] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.709764 20176 ts_tablet_manager.cc:1428] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260807 19:49:05.754091 20184 raft_consensus.cc:482] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:05.754519 20184 raft_consensus.cc:504] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.756681 20184 leader_election.cc:288] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44b5544790f3471bbde8a710747834fa (127.0.198.194:35677), 2632aff2d9514c7db14ac640e1495800 (127.0.198.195:46109)
I20260807 19:49:05.765343 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.757425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40569 (local address 127.0.198.195:46109)
0807 19:49:05.757667 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.757675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.758675 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 19:49:05.758815 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.758820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.758907 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.759021 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:05.759980 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.761036 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.763462 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.764297 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.764506 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.764595 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.764925 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.765083 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:49:05.765456 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.757336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:46109 (local address 127.0.198.193:40569)
0807 19:49:05.758445 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.758463 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.758656 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.759078 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.759084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.759118 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:05.759729 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.759753 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.761195 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.761202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.763184 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.763199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.764548 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.764555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.764727 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.764751 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.764892 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.765042 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":191,"thread_start_us":563,"threads_started":1}
I20260807 19:49:05.765712 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.757103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35677 (local address 127.0.198.193:40403)
0807 19:49:05.758413 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:05.758445 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:05.758698 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:05.759624 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:05.759629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:05.759659 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:05.760159 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.760173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.761269 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.761275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.763171 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:05.763189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.764479 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.764486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:05.764652 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.764678 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:49:05.764804 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:49:05.764956 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"mutex_wait_us":174,"spinlock_wait_cycles":26496,"thread_start_us":839,"threads_started":1}
I20260807 19:49:05.765272 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.757430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:40403 (local address 127.0.198.194:35677)
0807 19:49:05.757669 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.757676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.758675 (+ 999us) server_negotiation.cc:407] Connection header received
0807 19:49:05.759345 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.759377 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.759476 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.759589 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:05.760341 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.761104 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.763452 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.764252 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.764450 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.764526 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.764847 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.764985 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:49:05.767493 20106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ada4a978e84422faf25e4689560c167" candidate_uuid: "7bb8532eab24499dacdf2b5f64ef0580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2632aff2d9514c7db14ac640e1495800" is_pre_election: true
I20260807 19:49:05.767486 19972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ada4a978e84422faf25e4689560c167" candidate_uuid: "7bb8532eab24499dacdf2b5f64ef0580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b5544790f3471bbde8a710747834fa" is_pre_election: true
I20260807 19:49:05.768172 20106 raft_consensus.cc:2454] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bb8532eab24499dacdf2b5f64ef0580 in term 0.
I20260807 19:49:05.768172 19972 raft_consensus.cc:2454] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bb8532eab24499dacdf2b5f64ef0580 in term 0.
I20260807 19:49:05.769145 19766 leader_election.cc:302] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2632aff2d9514c7db14ac640e1495800, 44b5544790f3471bbde8a710747834fa, 7bb8532eab24499dacdf2b5f64ef0580; no voters:
I20260807 19:49:05.769759 20184 raft_consensus.cc:2781] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:05.769953 20184 raft_consensus.cc:482] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:05.770128 20184 raft_consensus.cc:3037] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:05.772886 20184 raft_consensus.cc:504] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.773728 20184 leader_election.cc:288] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [CANDIDATE]: Term 1 election: Requested vote from peers 44b5544790f3471bbde8a710747834fa (127.0.198.194:35677), 2632aff2d9514c7db14ac640e1495800 (127.0.198.195:46109)
I20260807 19:49:05.774384 20106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ada4a978e84422faf25e4689560c167" candidate_uuid: "7bb8532eab24499dacdf2b5f64ef0580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2632aff2d9514c7db14ac640e1495800"
I20260807 19:49:05.774425 19972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ada4a978e84422faf25e4689560c167" candidate_uuid: "7bb8532eab24499dacdf2b5f64ef0580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b5544790f3471bbde8a710747834fa"
I20260807 19:49:05.774643 20106 raft_consensus.cc:3037] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:05.774739 19972 raft_consensus.cc:3037] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:05.777287 20106 raft_consensus.cc:2454] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bb8532eab24499dacdf2b5f64ef0580 in term 1.
I20260807 19:49:05.777472 19972 raft_consensus.cc:2454] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bb8532eab24499dacdf2b5f64ef0580 in term 1.
I20260807 19:49:05.778031 19766 leader_election.cc:302] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2632aff2d9514c7db14ac640e1495800, 7bb8532eab24499dacdf2b5f64ef0580; no voters:
I20260807 19:49:05.778503 20184 raft_consensus.cc:2781] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:05.779076 20184 raft_consensus.cc:686] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 1 LEADER]: Becoming Leader. State: Replica: 7bb8532eab24499dacdf2b5f64ef0580, State: Running, Role: LEADER
I20260807 19:49:05.779706 20184 consensus_queue.cc:237] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } }
I20260807 19:49:05.785280 19687 catalog_manager.cc:5495] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bb8532eab24499dacdf2b5f64ef0580 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "7bb8532eab24499dacdf2b5f64ef0580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb8532eab24499dacdf2b5f64ef0580" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 43485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:05.803753 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.798018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.0.198.193:43485)
0807 19:49:05.798258 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.798264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.798287 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:49:05.798395 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.798399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.798544 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.798718 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:05.799121 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.799937 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.801398 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.801858 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.801998 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.802555 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:05.803093 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:05.803259 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.803388 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.803481 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:49:05.804757 19882 tablet.cc:2369] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:05.808923 20019 tablet.cc:2369] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:05.809903 20157 tablet.cc:2369] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:05.813243 20106 raft_consensus.cc:1261] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 1 FOLLOWER]: Refusing update from remote peer 7bb8532eab24499dacdf2b5f64ef0580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.813274 19972 raft_consensus.cc:1261] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 1 FOLLOWER]: Refusing update from remote peer 7bb8532eab24499dacdf2b5f64ef0580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.814577 20184 consensus_queue.cc:1035] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2632aff2d9514c7db14ac640e1495800" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 46109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:05.814968 20187 consensus_queue.cc:1035] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b5544790f3471bbde8a710747834fa" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:05.848078 19687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:05.848435 19687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:05.852861 19687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0b369d99dc443cfba195d763f70f89a: Sending DeleteTablet for 3 replicas of tablet 8ada4a978e84422faf25e4689560c167
I20260807 19:49:05.854341 20086 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ada4a978e84422faf25e4689560c167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:05 UTC) from {username='slave'} at 127.0.0.1:60302
I20260807 19:49:05.854414 19815 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ada4a978e84422faf25e4689560c167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:05 UTC) from {username='slave'} at 127.0.0.1:40566
I20260807 19:49:05.854748 19951 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ada4a978e84422faf25e4689560c167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:05 UTC) from {username='slave'} at 127.0.0.1:35708
I20260807 19:49:05.855597 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19750
I20260807 19:49:05.865975 20203 tablet_replica.cc:330] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: stopping tablet replica
I20260807 19:49:05.867895 20202 tablet_replica.cc:330] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: stopping tablet replica
I20260807 19:49:05.868482 20202 raft_consensus.cc:2229] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:05.869550 20203 raft_consensus.cc:2229] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:05.870136 20203 raft_consensus.cc:2258] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:05.870618 20202 raft_consensus.cc:2258] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:05.873572 20203 ts_tablet_manager.cc:1905] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:05.879819 20202 ts_tablet_manager.cc:1905] T 8ada4a978e84422faf25e4689560c167 P 7bb8532eab24499dacdf2b5f64ef0580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:05.881606 20201 tablet_replica.cc:330] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: stopping tablet replica
I20260807 19:49:05.882159 20201 raft_consensus.cc:2229] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:05.882251 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:05.857576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.0.198.193:43485)
0807 19:49:05.857826 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:05.857836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:05.869048 (+ 11212us) server_negotiation.cc:407] Connection header received
0807 19:49:05.869490 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:05.869500 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:05.869656 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:05.869761 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:05.871041 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.872797 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.875869 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:05.876895 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:05.877045 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:05.879659 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:05.879683 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:05.879710 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:05.880945 (+ 1235us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:05.881047 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:05.881088 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:05.881226 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:05.881658 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 19:49:05.881817 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:49:05.882807 20201 raft_consensus.cc:2258] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:05.883251 19876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:05.888576 20201 ts_tablet_manager.cc:1905] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:05.888934 20203 ts_tablet_manager.cc:1918] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:05.889142 20203 log.cc:1196] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/8ada4a978e84422faf25e4689560c167
I20260807 19:49:05.889791 20203 ts_tablet_manager.cc:1939] T 8ada4a978e84422faf25e4689560c167 P 44b5544790f3471bbde8a710747834fa: Deleting consensus metadata
I20260807 19:49:05.892046 19671 catalog_manager.cc:4841] TS 44b5544790f3471bbde8a710747834fa (127.0.198.194:35677): tablet 8ada4a978e84422faf25e4689560c167 (table test-table [id=78e9773bb5964c1d8d2753b08d55e75a]) successfully deleted
I20260807 19:49:05.895279 20201 ts_tablet_manager.cc:1918] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:05.895526 20201 log.cc:1196] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/8ada4a978e84422faf25e4689560c167
I20260807 19:49:05.896023 20201 ts_tablet_manager.cc:1939] T 8ada4a978e84422faf25e4689560c167 P 2632aff2d9514c7db14ac640e1495800: Deleting consensus metadata
I20260807 19:49:05.897971 19672 catalog_manager.cc:4841] TS 2632aff2d9514c7db14ac640e1495800 (127.0.198.195:46109): tablet 8ada4a978e84422faf25e4689560c167 (table test-table [id=78e9773bb5964c1d8d2753b08d55e75a]) successfully deleted
I20260807 19:49:05.987473 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19750
I20260807 19:49:06.009445 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19885
W20260807 19:49:06.009554 19671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:43485 (error 108)
I20260807 19:49:06.012018 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.011390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40578)
0807 19:49:06.011591 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.011815 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:49:06.012305 19671 catalog_manager.cc:4584] TS 7bb8532eab24499dacdf2b5f64ef0580 (127.0.198.193:43485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ada4a978e84422faf25e4689560c167: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
I20260807 19:49:06.017611 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.010891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.0.198.194:35677)
0807 19:49:06.011126 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:06.011132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:06.011153 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:49:06.011223 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:06.011228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:06.011448 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:06.011571 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:06.012118 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:06.013090 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:06.014636 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:06.015051 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:06.015148 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:06.016349 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:06.016359 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:06.016373 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:06.016960 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:06.017009 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:06.017034 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:06.017122 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:06.017251 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:49:06.017343 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:49:06.018458 20013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:06.066232 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.065620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40592)
0807 19:49:06.065905 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.066025 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 19:49:06.067608 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.067105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40608)
0807 19:49:06.067289 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.067454 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:49:06.113586 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19885
I20260807 19:49:06.128657 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.128115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40612)
0807 19:49:06.128368 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.128467 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 19:49:06.129853 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.129488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40620)
0807 19:49:06.129642 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.129723 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 19:49:06.134639 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20021
I20260807 19:49:06.142408 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.135551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.0.198.195:46109)
0807 19:49:06.135721 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:06.135726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:06.135836 (+ 110us) server_negotiation.cc:407] Connection header received
0807 19:49:06.135997 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:06.136002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:06.136104 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:06.136197 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:06.136771 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:06.137577 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:06.139104 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:06.139687 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:06.139792 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:06.140977 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:06.140986 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:06.141002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:06.141679 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:06.141738 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:06.141761 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:06.141859 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:06.142044 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 19:49:06.142154 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:49:06.143141 20149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:06.211112 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.210465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40624)
0807 19:49:06.210746 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.210922 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 19:49:06.212515 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:06.212065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:43485 (local address 127.0.0.1:40634)
0807 19:49:06.212303 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:06.212396 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 19:49:06.237758 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20021
I20260807 19:49:06.257684 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 19656
I20260807 19:49:06.259064 19719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:06.375314 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 19656
2026-08-07T19:49:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3026 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:06 dist-test-slave-btw8 krb5kdc[20212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:06 dist-test-slave-btw8 krb5kdc[20212](info): set up 2 sockets
Aug 07 19:49:06 dist-test-slave-btw8 krb5kdc[20212](info): commencing operation
krb5kdc: starting...
W20260807 19:49:07.576872 795 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:07 dist-test-slave-btw8 krb5kdc[20212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:49:07.944520 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33473
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:08.315615 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:08.316031 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:08.316197 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:08.325012 20227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:08.325227 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:08.325368 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:08.325480 20227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:08.325584 20227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:08.337246 20227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:08.338835 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:08.340396 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:08.355001 20232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:08.354974 20235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:08.354945 20233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:08.356547 20227 server_base.cc:1034] running on GCE node
I20260807 19:49:08.357648 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:08.359652 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:08.361007 20227 hybrid_clock.cc:648] HybridClock initialized: now 1786132148360954 us; error 61 us; skew 500 ppm
I20260807 19:49:08.361562 20227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:08 dist-test-slave-btw8 krb5kdc[20212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:08.369674 20227 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:08.371797 20227 webserver.cc:460] Webserver started at http://127.0.198.254:38471/ using document root <none> and password file <none>
I20260807 19:49:08.372704 20227 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:08.372886 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:08.373274 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:08.377043 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "68952af0551a4d61930599ad9f937f70"
format_stamp: "Formatted at 2026-08-07 19:49:08 on dist-test-slave-btw8"
server_key: "af8bf003ec60c9fefa776d78f02d0ab0"
server_key_iv: "8ca44629383ef10902b6068fce5e0d15"
server_key_version: "encryptionkey@0"
I20260807 19:49:08.377980 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "68952af0551a4d61930599ad9f937f70"
format_stamp: "Formatted at 2026-08-07 19:49:08 on dist-test-slave-btw8"
server_key: "af8bf003ec60c9fefa776d78f02d0ab0"
server_key_iv: "8ca44629383ef10902b6068fce5e0d15"
server_key_version: "encryptionkey@0"
I20260807 19:49:08.384824 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:49:08.389247 20242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:08.390851 20227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:49:08.391098 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "68952af0551a4d61930599ad9f937f70"
format_stamp: "Formatted at 2026-08-07 19:49:08 on dist-test-slave-btw8"
server_key: "af8bf003ec60c9fefa776d78f02d0ab0"
server_key_iv: "8ca44629383ef10902b6068fce5e0d15"
server_key_version: "encryptionkey@0"
I20260807 19:49:08.391414 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:08.417650 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:08.423918 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:08.424268 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:08.446357 20227 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33473
I20260807 19:49:08.446357 20298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33473 every 8 connection(s)
I20260807 19:49:08.448194 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:08.452862 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20227
I20260807 19:49:08.453158 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:08.453308 20299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:08.453800 795 external_mini_cluster.cc:1428] Setting key 85a1da29c64ae3d4d05d4752da07209a
I20260807 19:49:08.471303 20299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: Bootstrap starting.
Aug 07 19:49:08 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:08.476917 20299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:08.478840 20299 log.cc:824] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:08.484005 20299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: No bootstrap required, opened a new log
I20260807 19:49:08.486687 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:08.456015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.0.198.254:33473)
0807 19:49:08.457612 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:08.457626 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:08.457676 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:49:08.459470 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:08.459499 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:08.460023 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:08.460438 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:08.462957 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:08.464536 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:08.466935 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:08.467956 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:08.468155 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:08.475669 (+ 7514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:08.475683 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:08.475711 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:08.475785 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:08.480972 (+ 5187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:08.482271 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:08.482281 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:08.482292 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:08.482549 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:08.483090 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:08.483095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:08.483098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:08.483890 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:08.484371 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:08.484685 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 19:49:08.485207 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1317,"thread_start_us":881,"threads_started":1}
I20260807 19:49:08.494127 20299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } }
I20260807 19:49:08.494608 20299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:08.494690 20299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68952af0551a4d61930599ad9f937f70, State: Initialized, Role: FOLLOWER
I20260807 19:49:08.495513 20299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } }
I20260807 19:49:08.495842 20299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:08.496006 20299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:08.496240 20299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:08.499222 20299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } }
I20260807 19:49:08.499951 20299 leader_election.cc:302] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68952af0551a4d61930599ad9f937f70; no voters:
I20260807 19:49:08.500641 20299 leader_election.cc:288] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:08.500818 20305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:08.501542 20305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [term 1 LEADER]: Becoming Leader. State: Replica: 68952af0551a4d61930599ad9f937f70, State: Running, Role: LEADER
I20260807 19:49:08.502216 20305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } }
I20260807 19:49:08.502969 20299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:08.505246 20307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68952af0551a4d61930599ad9f937f70. Latest consensus state: current_term: 1 leader_uuid: "68952af0551a4d61930599ad9f937f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } } }
I20260807 19:49:08.505731 20307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:08.506214 20306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68952af0551a4d61930599ad9f937f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68952af0551a4d61930599ad9f937f70" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33473 } } }
I20260807 19:49:08.506915 20306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:08.516901 20312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:08.521806 20312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:08.531677 20312 catalog_manager.cc:1347] Generated new cluster ID: c6003baccebe42c4a1265664f52b2b84
I20260807 19:49:08.531935 20312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:08.546440 20312 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:08.548614 20312 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:08.563932 20312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: Generated new TSK 0
I20260807 19:49:08.564966 20312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:49:08.656186 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:09.044485 20326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:09.044884 20326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:09.045069 20326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:09.052870 20326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:09.053078 20326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:09.053284 20326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:09.064356 20326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:09.065855 20326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:09.067461 20326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:09.082858 20332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:09.083029 20331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:09.083027 20334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:09.084018 20326 server_base.cc:1034] running on GCE node
I20260807 19:49:09.084811 20326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:09.085968 20326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:09.087283 20326 hybrid_clock.cc:648] HybridClock initialized: now 1786132149087222 us; error 59 us; skew 500 ppm
I20260807 19:49:09.087721 20326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:09 dist-test-slave-btw8 krb5kdc[20212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:09.091583 20326 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:49:09.093266 20326 webserver.cc:460] Webserver started at http://127.0.198.193:38425/ using document root <none> and password file <none>
I20260807 19:49:09.093976 20326 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:09.094096 20326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:09.094391 20326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:09.097864 20326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "150d160fe3ce4471919692c8c50e78c7"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "f1753782b15024a0e2781be7da05b26a"
server_key_iv: "a69456fe972169a69788e1b1113ff8b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.098675 20326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "150d160fe3ce4471919692c8c50e78c7"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "f1753782b15024a0e2781be7da05b26a"
server_key_iv: "a69456fe972169a69788e1b1113ff8b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.104470 20326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:49:09.108740 20341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.110473 20326 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 19:49:09.110646 20326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "150d160fe3ce4471919692c8c50e78c7"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "f1753782b15024a0e2781be7da05b26a"
server_key_iv: "a69456fe972169a69788e1b1113ff8b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.110855 20326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:09.132874 20326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:09.139093 20326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:09.139591 20326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:09.140857 20326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:09.142556 20326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:09.142650 20326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.142765 20326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:09.142822 20326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.184440 20326 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:35375
I20260807 19:49:09.184679 20459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:35375 every 8 connection(s)
I20260807 19:49:09.186132 20326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:09.188385 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20326
I20260807 19:49:09.188623 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:09.189320 795 external_mini_cluster.cc:1428] Setting key db5f1da89b7a0e8ac85231cdf02f9840
Aug 07 19:49:09 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:09.210041 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:09.189269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:52111 (local address 127.0.198.254:33473)
0807 19:49:09.189776 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:09.189780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:09.191002 (+ 1222us) server_negotiation.cc:407] Connection header received
0807 19:49:09.193557 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:09.193563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:09.193660 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:09.193796 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:09.195469 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.196344 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.198253 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.198692 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.198847 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:09.202934 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:09.202940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:09.202944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:09.202981 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:09.205910 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:09.206966 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:09.206976 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:09.206980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:09.207197 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:09.207900 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:09.207904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:09.207907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:09.208261 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:09.208390 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:09.209544 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0807 19:49:09.209771 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":267,"threads_started":1}
I20260807 19:49:09.211728 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:09.189987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33473 (local address 127.0.198.193:52111)
0807 19:49:09.190853 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:09.190894 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:09.193252 (+ 2358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:09.194070 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:09.194080 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:09.194646 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:09.195257 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:09.195276 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.196517 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.196524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:09.198075 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:09.198091 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.198973 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.198982 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:09.199269 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:09.200117 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:09.200147 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:09.202553 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:09.206089 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:09.206100 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:09.206128 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:09.206644 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:09.207393 (+ 749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:09.207401 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:09.207406 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:09.207731 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:09.208405 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:09.208416 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:09.209043 (+ 627us) client_negotiation.cc:769] Sending connection context
0807 19:49:09.209391 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 19:49:09.210046 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":340,"threads_started":1}
I20260807 19:49:09.213667 20460 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33473
I20260807 19:49:09.214128 20460 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:09.215168 20460 heartbeater.cc:511] Master 127.0.198.254:33473 requested a full tablet report, sending...
I20260807 19:49:09.217845 20260 ts_manager.cc:194] Registered new tserver with Master: 150d160fe3ce4471919692c8c50e78c7 (127.0.198.193:35375)
I20260807 19:49:09.219528 20260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:52111
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:49:09.272136 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:09.662003 20466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:09.662375 20466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:09.662545 20466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:09.670346 20466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:09.670534 20466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:09.670724 20466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:09.682364 20466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:09.684095 20466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:09.685767 20466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:09.702123 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:09.702167 20474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:09.702370 20471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:09.703887 20466 server_base.cc:1034] running on GCE node
I20260807 19:49:09.704711 20466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:09.705897 20466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:09.707152 20466 hybrid_clock.cc:648] HybridClock initialized: now 1786132149707101 us; error 59 us; skew 500 ppm
I20260807 19:49:09.707634 20466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:09 dist-test-slave-btw8 krb5kdc[20212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:09.711484 20466 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:49:09.713394 20466 webserver.cc:460] Webserver started at http://127.0.198.194:35705/ using document root <none> and password file <none>
I20260807 19:49:09.714334 20466 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:09.714510 20466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:09.714926 20466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:09.718184 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "9785e43b9a024417ad9cdf4f9a358d7c"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "6bee29f19cba012f6767c4012f492c76"
server_key_iv: "78fba2f88789038afd33cb496afdc5f6"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.719218 20466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "9785e43b9a024417ad9cdf4f9a358d7c"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "6bee29f19cba012f6767c4012f492c76"
server_key_iv: "78fba2f88789038afd33cb496afdc5f6"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.725301 20466 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:09.729534 20481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.731089 20466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:49:09.731400 20466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "9785e43b9a024417ad9cdf4f9a358d7c"
format_stamp: "Formatted at 2026-08-07 19:49:09 on dist-test-slave-btw8"
server_key: "6bee29f19cba012f6767c4012f492c76"
server_key_iv: "78fba2f88789038afd33cb496afdc5f6"
server_key_version: "encryptionkey@0"
I20260807 19:49:09.731743 20466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:09.764550 20466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:09.769474 20466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:09.769819 20466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:09.771054 20466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:09.772694 20466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:09.772818 20466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.772982 20466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:09.773067 20466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:09.810105 20466 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:37069
I20260807 19:49:09.810271 20599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:37069 every 8 connection(s)
I20260807 19:49:09.811549 20466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:09.820647 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20466
I20260807 19:49:09.820912 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:09.821590 795 external_mini_cluster.cc:1428] Setting key 41c403dbb6902b054d4dee2b0563065c
Aug 07 19:49:09 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:09.835702 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:09.814322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:47135 (local address 127.0.198.254:33473)
0807 19:49:09.814854 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:09.814859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:09.816083 (+ 1224us) server_negotiation.cc:407] Connection header received
0807 19:49:09.818035 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:09.818040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:09.818143 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:09.818271 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:09.819962 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.820722 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.823690 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.824362 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.824475 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:09.829262 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:09.829267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:09.829271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:09.829308 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:09.832186 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:09.833069 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:09.833073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:09.833076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:09.833218 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:09.833721 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:09.833725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:09.833727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:09.834055 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:09.834236 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:09.835008 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 19:49:09.835268 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
I20260807 19:49:09.836911 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:09.815060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33473 (local address 127.0.198.194:47135)
0807 19:49:09.815899 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:09.815963 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:09.817735 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:09.818538 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:09.818546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:09.819086 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:09.819766 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:09.819785 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.821033 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.821041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:09.823457 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:09.823479 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:09.824671 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:09.824681 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:09.825037 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:09.825920 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:09.825958 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:09.828945 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:09.832376 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:09.832387 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:09.832408 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:09.832907 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:09.833349 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:09.833354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:09.833358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:09.833579 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:09.834199 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:09.834209 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:09.834611 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 19:49:09.834907 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 19:49:09.835572 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":355,"threads_started":1}
I20260807 19:49:09.838920 20600 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33473
I20260807 19:49:09.839340 20600 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:09.840292 20600 heartbeater.cc:511] Master 127.0.198.254:33473 requested a full tablet report, sending...
I20260807 19:49:09.842353 20260 ts_manager.cc:194] Registered new tserver with Master: 9785e43b9a024417ad9cdf4f9a358d7c (127.0.198.194:37069)
I20260807 19:49:09.843442 20260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:47135
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:49:09.903046 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:49:10.223464 20460 heartbeater.cc:503] Master 127.0.198.254:33473 was elected leader, sending a full tablet report...
W20260807 19:49:10.271924 20606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:10.272297 20606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:10.272477 20606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:10.280089 20606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:10.280264 20606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:10.280463 20606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:10.292169 20606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:10.293685 20606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:10.295159 20606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:10.309621 20613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:10.309672 20615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:10.309733 20612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:10.310819 20606 server_base.cc:1034] running on GCE node
I20260807 19:49:10.311659 20606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:10.312795 20606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:10.314025 20606 hybrid_clock.cc:648] HybridClock initialized: now 1786132150313949 us; error 65 us; skew 500 ppm
I20260807 19:49:10.314374 20606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:10 dist-test-slave-btw8 krb5kdc[20212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:10.317971 20606 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:49:10.319772 20606 webserver.cc:460] Webserver started at http://127.0.198.195:40173/ using document root <none> and password file <none>
I20260807 19:49:10.320499 20606 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:10.320627 20606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:10.320914 20606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:10.324504 20606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "8cb5e871046f461f917e4345fc3d89a1"
format_stamp: "Formatted at 2026-08-07 19:49:10 on dist-test-slave-btw8"
server_key: "3c821346a10e7ff2936bb56f4d498648"
server_key_iv: "a3467e801f90e4ce4c992272e712c664"
server_key_version: "encryptionkey@0"
I20260807 19:49:10.325356 20606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "8cb5e871046f461f917e4345fc3d89a1"
format_stamp: "Formatted at 2026-08-07 19:49:10 on dist-test-slave-btw8"
server_key: "3c821346a10e7ff2936bb56f4d498648"
server_key_iv: "a3467e801f90e4ce4c992272e712c664"
server_key_version: "encryptionkey@0"
I20260807 19:49:10.331204 20606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:49:10.335479 20622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:10.337337 20606 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 19:49:10.337580 20606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "8cb5e871046f461f917e4345fc3d89a1"
format_stamp: "Formatted at 2026-08-07 19:49:10 on dist-test-slave-btw8"
server_key: "3c821346a10e7ff2936bb56f4d498648"
server_key_iv: "a3467e801f90e4ce4c992272e712c664"
server_key_version: "encryptionkey@0"
I20260807 19:49:10.337843 20606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:10.377648 20606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:10.382676 20606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:10.382969 20606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:10.384167 20606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:10.385990 20606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:10.386080 20606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:10.386169 20606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:10.386214 20606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:10.424322 20606 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:44077
I20260807 19:49:10.424466 20739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:44077 every 8 connection(s)
I20260807 19:49:10.425783 20606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:10.433265 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20606
I20260807 19:49:10.433468 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:10.434007 795 external_mini_cluster.cc:1428] Setting key 16a8396c8b2455d8b9419f456763ac62
Aug 07 19:49:10 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:10.451624 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.428988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:53309 (local address 127.0.198.254:33473)
0807 19:49:10.429576 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.429581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.430468 (+ 887us) server_negotiation.cc:407] Connection header received
0807 19:49:10.432883 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.432890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.433015 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.433172 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:10.435333 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.436633 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.438655 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.439205 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.439404 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.443622 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:10.443630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:10.443635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:10.443681 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:10.447143 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.448557 (+ 1414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.448563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.448567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:10.448736 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.449367 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.449374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.449382 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:10.449800 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:10.449993 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.450918 (+ 925us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.451156 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":324,"threads_started":1}
I20260807 19:49:10.452816 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.429484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:33473 (local address 127.0.198.195:53309)
0807 19:49:10.430283 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.430329 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.432489 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.433546 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.433564 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.434264 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:10.435096 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.435119 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.436826 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.436830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.438462 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.438480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.439506 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.439515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.439815 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.440551 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:10.440668 (+ 117us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:10.443294 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:10.447455 (+ 4161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:10.447465 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:10.447485 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:10.448396 (+ 911us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:10.448871 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:10.448876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:10.448880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:10.449222 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:10.449955 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:10.449964 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:10.450448 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.450752 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.451408 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":331,"threads_started":1}
I20260807 19:49:10.454423 20740 heartbeater.cc:347] Connected to a master server at 127.0.198.254:33473
I20260807 19:49:10.454860 20740 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:10.455757 20740 heartbeater.cc:511] Master 127.0.198.254:33473 requested a full tablet report, sending...
I20260807 19:49:10.457474 20260 ts_manager.cc:194] Registered new tserver with Master: 8cb5e871046f461f917e4345fc3d89a1 (127.0.198.195:44077)
I20260807 19:49:10.458400 20260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:53309
I20260807 19:49:10.464694 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:10.486291 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.469841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.0.198.254:33473)
0807 19:49:10.470061 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.470066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.470487 (+ 421us) server_negotiation.cc:407] Connection header received
0807 19:49:10.470596 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.470604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.470710 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.470834 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:10.472481 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.473365 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.474749 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.475270 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.475498 (+ 228us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.479183 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:10.479190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:10.479194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:10.479239 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:10.482427 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.483582 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.483586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.483589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:10.483824 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.484466 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.484470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.484472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:10.484959 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:10.485210 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.485811 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.486014 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:49:10.491492 20260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:10.499037 20260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:10.537967 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.514410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:44077 (local address 127.0.0.1:38300)
0807 19:49:10.527077 (+ 12667us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.527103 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.527404 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.528362 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.528373 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.528424 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:10.529364 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.529386 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.531231 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.531236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.534302 (+ 3066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.534322 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.536850 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.536856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.537099 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.537174 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.537334 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.537564 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12529,"mutex_wait_us":147,"thread_start_us":14302,"threads_started":1}
I20260807 19:49:10.538765 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.514865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47874)
0807 19:49:10.529217 (+ 14352us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.529293 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.529452 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.530313 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.530318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.530346 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:10.530708 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.530720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.532626 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.532633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.534621 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.534639 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.537631 (+ 2992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.537636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.537979 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.538043 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.538151 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.538295 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14149,"mutex_wait_us":129,"thread_start_us":12085,"threads_started":1}
I20260807 19:49:10.538872 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.515134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.0.198.195:44077)
0807 19:49:10.525837 (+ 10703us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.525932 (+ 95us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.527241 (+ 1309us) server_negotiation.cc:407] Connection header received
0807 19:49:10.527631 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.527643 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.527926 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.528201 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:10.529618 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.531071 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.534531 (+ 3460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.536216 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.536383 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.536711 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.538280 (+ 1569us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.538504 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10340,"thread_start_us":9868,"threads_started":1}
I20260807 19:49:10.539191 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.514489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.0.198.194:37069)
0807 19:49:10.527650 (+ 13161us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.527659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.527684 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:49:10.527792 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.527799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.527973 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.528318 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:10.529678 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.531249 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.534666 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.536994 (+ 2328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.537166 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.537378 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.538560 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.538829 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12986,"thread_start_us":12834,"threads_started":1}
I20260807 19:49:10.539578 20753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.514159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:37069 (local address 127.0.0.1:36934)
0807 19:49:10.526368 (+ 12209us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.526458 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.526783 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.528577 (+ 1794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.528584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.528626 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:10.529450 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.529460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.531449 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.531456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.534451 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.534468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.537624 (+ 3156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.537633 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.537997 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.538375 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.538482 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.538633 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11815,"mutex_wait_us":562,"thread_start_us":11594,"threads_started":1}
I20260807 19:49:10.539757 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.515733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.0.198.193:35375)
0807 19:49:10.526583 (+ 10850us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.526591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.529622 (+ 3031us) server_negotiation.cc:407] Connection header received
0807 19:49:10.529742 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.529752 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.530048 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.530288 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:10.530889 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.532473 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.534828 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.536994 (+ 2166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.537167 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.537419 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.539120 (+ 1701us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.539342 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10632,"thread_start_us":10940,"threads_started":1}
I20260807 19:49:10.544063 20392 tablet_service.cc:1467] Processing CreateTablet for tablet de20ad34f6824d22b77105e24962df12 (DEFAULT_TABLE table=test-table [id=6d31c5d73bfb4d0cb4fc697e1079b99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:10.544140 20670 tablet_service.cc:1467] Processing CreateTablet for tablet de20ad34f6824d22b77105e24962df12 (DEFAULT_TABLE table=test-table [id=6d31c5d73bfb4d0cb4fc697e1079b99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:10.544355 20532 tablet_service.cc:1467] Processing CreateTablet for tablet de20ad34f6824d22b77105e24962df12 (DEFAULT_TABLE table=test-table [id=6d31c5d73bfb4d0cb4fc697e1079b99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:10.545765 20392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de20ad34f6824d22b77105e24962df12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:10.545840 20670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de20ad34f6824d22b77105e24962df12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:10.545832 20532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de20ad34f6824d22b77105e24962df12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:10.560189 20759 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Bootstrap starting.
I20260807 19:49:10.564142 20761 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Bootstrap starting.
I20260807 19:49:10.566386 20760 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Bootstrap starting.
I20260807 19:49:10.568408 20759 tablet_bootstrap.cc:654] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:10.569698 20761 tablet_bootstrap.cc:654] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:10.570215 20760 tablet_bootstrap.cc:654] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:10.570283 20759 log.cc:824] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:10.571085 20761 log.cc:824] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:10.571424 20760 log.cc:824] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:10.574892 20761 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: No bootstrap required, opened a new log
I20260807 19:49:10.574892 20759 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: No bootstrap required, opened a new log
I20260807 19:49:10.575219 20759 ts_tablet_manager.cc:1397] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.009s
I20260807 19:49:10.575219 20761 ts_tablet_manager.cc:1397] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260807 19:49:10.577162 20760 tablet_bootstrap.cc:492] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: No bootstrap required, opened a new log
I20260807 19:49:10.577498 20760 ts_tablet_manager.cc:1397] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260807 19:49:10.586287 20760 raft_consensus.cc:348] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.586833 20760 raft_consensus.cc:374] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:10.586971 20760 raft_consensus.cc:729] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb5e871046f461f917e4345fc3d89a1, State: Initialized, Role: FOLLOWER
I20260807 19:49:10.587283 20761 raft_consensus.cc:348] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.587283 20759 raft_consensus.cc:348] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.587811 20759 raft_consensus.cc:374] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:10.587800 20761 raft_consensus.cc:374] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:10.587921 20761 raft_consensus.cc:729] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9785e43b9a024417ad9cdf4f9a358d7c, State: Initialized, Role: FOLLOWER
I20260807 19:49:10.587917 20759 raft_consensus.cc:729] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150d160fe3ce4471919692c8c50e78c7, State: Initialized, Role: FOLLOWER
I20260807 19:49:10.588065 20760 consensus_queue.cc:260] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.588688 20761 consensus_queue.cc:260] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.588685 20759 consensus_queue.cc:260] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.589566 20740 heartbeater.cc:503] Master 127.0.198.254:33473 was elected leader, sending a full tablet report...
I20260807 19:49:10.590265 20760 ts_tablet_manager.cc:1428] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 19:49:10.591933 20600 heartbeater.cc:503] Master 127.0.198.254:33473 was elected leader, sending a full tablet report...
I20260807 19:49:10.592243 20759 ts_tablet_manager.cc:1428] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260807 19:49:10.593768 20761 ts_tablet_manager.cc:1428] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Time spent starting tablet: real 0.018s user 0.014s sys 0.005s
W20260807 19:49:10.677865 20741 tablet.cc:2369] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:10.689456 20461 tablet.cc:2369] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:10.813971 20601 tablet.cc:2369] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:10.858052 20765 raft_consensus.cc:482] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:10.858352 20765 raft_consensus.cc:504] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.860185 20765 leader_election.cc:288] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9785e43b9a024417ad9cdf4f9a358d7c (127.0.198.194:37069), 150d160fe3ce4471919692c8c50e78c7 (127.0.198.193:35375)
I20260807 19:49:10.867929 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.860746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:42121 (local address 127.0.198.194:37069)
0807 19:49:10.860946 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.860952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.860974 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:49:10.861175 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.861181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.861270 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.861393 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:10.862052 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.863114 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.865638 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.866839 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.867027 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.867113 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.867495 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.867663 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:49:10.867986 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.860651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:37069 (local address 127.0.198.195:42121)
0807 19:49:10.860831 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.860853 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.860998 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.861460 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.861467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.861520 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:10.861832 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.861851 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.863281 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.863284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.865426 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.865442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.867017 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.867021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.867262 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.867289 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.867447 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.867649 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":57}
I20260807 19:49:10.868521 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.860763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:35433 (local address 127.0.198.193:35375)
0807 19:49:10.860971 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.860976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.862007 (+ 1031us) server_negotiation.cc:407] Connection header received
0807 19:49:10.862098 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.862107 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.862196 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.862306 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:10.862789 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.863750 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.866069 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.867528 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.867699 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.867774 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.868151 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.868320 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:49:10.869381 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.860653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.198.195:35433)
0807 19:49:10.861860 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:10.861879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:10.862007 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:10.862323 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:10.862325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:10.862344 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:10.862625 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.862632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.863912 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.863917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.865872 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:10.865883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.867733 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.867738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:10.867918 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.867949 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:49:10.868822 (+ 873us) client_negotiation.cc:240] Negotiation successful
0807 19:49:10.868984 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":461,"thread_start_us":303,"threads_started":1}
I20260807 19:49:10.869248 20552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de20ad34f6824d22b77105e24962df12" candidate_uuid: "8cb5e871046f461f917e4345fc3d89a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" is_pre_election: true
I20260807 19:49:10.869938 20552 raft_consensus.cc:2454] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb5e871046f461f917e4345fc3d89a1 in term 0.
I20260807 19:49:10.870656 20412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de20ad34f6824d22b77105e24962df12" candidate_uuid: "8cb5e871046f461f917e4345fc3d89a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150d160fe3ce4471919692c8c50e78c7" is_pre_election: true
I20260807 19:49:10.870797 20626 leader_election.cc:302] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cb5e871046f461f917e4345fc3d89a1, 9785e43b9a024417ad9cdf4f9a358d7c; no voters:
I20260807 19:49:10.871251 20765 raft_consensus.cc:2781] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:10.871196 20412 raft_consensus.cc:2454] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cb5e871046f461f917e4345fc3d89a1 in term 0.
I20260807 19:49:10.871457 20765 raft_consensus.cc:482] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:10.871573 20765 raft_consensus.cc:3037] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:10.874154 20765 raft_consensus.cc:504] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.874912 20765 leader_election.cc:288] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [CANDIDATE]: Term 1 election: Requested vote from peers 9785e43b9a024417ad9cdf4f9a358d7c (127.0.198.194:37069), 150d160fe3ce4471919692c8c50e78c7 (127.0.198.193:35375)
I20260807 19:49:10.875670 20552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de20ad34f6824d22b77105e24962df12" candidate_uuid: "8cb5e871046f461f917e4345fc3d89a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9785e43b9a024417ad9cdf4f9a358d7c"
I20260807 19:49:10.875749 20412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de20ad34f6824d22b77105e24962df12" candidate_uuid: "8cb5e871046f461f917e4345fc3d89a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150d160fe3ce4471919692c8c50e78c7"
I20260807 19:49:10.875942 20552 raft_consensus.cc:3037] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:10.875968 20412 raft_consensus.cc:3037] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:10.879504 20552 raft_consensus.cc:2454] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cb5e871046f461f917e4345fc3d89a1 in term 1.
I20260807 19:49:10.879532 20412 raft_consensus.cc:2454] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cb5e871046f461f917e4345fc3d89a1 in term 1.
I20260807 19:49:10.880194 20624 leader_election.cc:302] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 150d160fe3ce4471919692c8c50e78c7, 8cb5e871046f461f917e4345fc3d89a1; no voters:
I20260807 19:49:10.880650 20765 raft_consensus.cc:2781] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:10.881138 20765 raft_consensus.cc:686] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 1 LEADER]: Becoming Leader. State: Replica: 8cb5e871046f461f917e4345fc3d89a1, State: Running, Role: LEADER
I20260807 19:49:10.881669 20765 consensus_queue.cc:237] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } }
I20260807 19:49:10.886744 20259 catalog_manager.cc:5495] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cb5e871046f461f917e4345fc3d89a1 (127.0.198.195). New cstate: current_term: 1 leader_uuid: "8cb5e871046f461f917e4345fc3d89a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cb5e871046f461f917e4345fc3d89a1" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 44077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:10.917059 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.910630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.0.198.195:44077)
0807 19:49:10.910943 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.910948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.910969 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:49:10.911088 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.911093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.911178 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.911345 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:10.911965 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.912714 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.914286 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.914787 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.914889 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.915658 (+ 769us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:10.916303 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:10.916447 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:10.916630 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:49:10.916674 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260807 19:49:10.926200 20552 raft_consensus.cc:1261] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 1 FOLLOWER]: Refusing update from remote peer 8cb5e871046f461f917e4345fc3d89a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:10.926189 20412 raft_consensus.cc:1261] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 1 FOLLOWER]: Refusing update from remote peer 8cb5e871046f461f917e4345fc3d89a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:10.927686 20765 consensus_queue.cc:1035] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "150d160fe3ce4471919692c8c50e78c7" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:10.928040 20769 consensus_queue.cc:1035] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9785e43b9a024417ad9cdf4f9a358d7c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 37069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:10.941798 20772 mvcc.cc:205] Tried to move back new op lower bound from 7315997290182381568 to 7315997290015494144. Current Snapshot: MvccSnapshot[applied={T|T < 7315997290182381568}]
I20260807 19:49:10.953830 20775 mvcc.cc:205] Tried to move back new op lower bound from 7315997290182381568 to 7315997290015494144. Current Snapshot: MvccSnapshot[applied={T|T < 7315997290182381568}]
I20260807 19:49:10.958482 20258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:10.958815 20258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:10.963469 20258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68952af0551a4d61930599ad9f937f70: Sending DeleteTablet for 3 replicas of tablet de20ad34f6824d22b77105e24962df12
I20260807 19:49:10.964991 20392 tablet_service.cc:1514] Processing DeleteTablet for tablet de20ad34f6824d22b77105e24962df12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:10 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:47874
I20260807 19:49:10.964998 20670 tablet_service.cc:1514] Processing DeleteTablet for tablet de20ad34f6824d22b77105e24962df12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:10 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260807 19:49:10.964986 20532 tablet_service.cc:1514] Processing DeleteTablet for tablet de20ad34f6824d22b77105e24962df12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:10 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:36934
I20260807 19:49:10.967450 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20326
I20260807 19:49:10.976213 20785 tablet_replica.cc:330] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: stopping tablet replica
I20260807 19:49:10.978075 20785 raft_consensus.cc:2229] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:10.980021 20785 raft_consensus.cc:2258] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:10.983870 20785 ts_tablet_manager.cc:1905] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:10.986166 20783 tablet_replica.cc:330] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: stopping tablet replica
I20260807 19:49:10.986405 20784 tablet_replica.cc:330] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: stopping tablet replica
I20260807 19:49:10.986908 20783 raft_consensus.cc:2229] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:10.986893 20784 raft_consensus.cc:2229] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:10.987545 20784 raft_consensus.cc:2258] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:10.987771 20783 raft_consensus.cc:2258] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:10.990473 20783 ts_tablet_manager.cc:1905] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:49:10 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:49:10.992869 20784 ts_tablet_manager.cc:1905] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:10.995626 20785 ts_tablet_manager.cc:1918] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:10.995846 20785 log.cc:1196] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/de20ad34f6824d22b77105e24962df12
I20260807 19:49:10.996475 20785 ts_tablet_manager.cc:1939] T de20ad34f6824d22b77105e24962df12 P 9785e43b9a024417ad9cdf4f9a358d7c: Deleting consensus metadata
I20260807 19:49:10.998572 20246 catalog_manager.cc:4841] TS 9785e43b9a024417ad9cdf4f9a358d7c (127.0.198.194:37069): tablet de20ad34f6824d22b77105e24962df12 (table test-table [id=6d31c5d73bfb4d0cb4fc697e1079b99c]) successfully deleted
I20260807 19:49:11.002312 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:10.969122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47888 (local address 127.0.198.193:35375)
0807 19:49:10.969385 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:10.969396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:10.978986 (+ 9590us) server_negotiation.cc:407] Connection header received
0807 19:49:10.979216 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:10.979222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:10.979318 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:10.979452 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:10.981894 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.983156 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.985362 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:10.986224 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:10.986371 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:10.993054 (+ 6683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:10.993076 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:10.993095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:10.993147 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:10.997786 (+ 4639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.999164 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.999176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.999180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:10.999413 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:10.999957 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:10.999963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:10.999966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:11.000449 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:11.000792 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:11.001519 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 19:49:11.001839 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:49:11.004112 20454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:11.004148 20783 ts_tablet_manager.cc:1918] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:11.004369 20783 log.cc:1196] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/de20ad34f6824d22b77105e24962df12
I20260807 19:49:11.004962 20783 ts_tablet_manager.cc:1939] T de20ad34f6824d22b77105e24962df12 P 8cb5e871046f461f917e4345fc3d89a1: Deleting consensus metadata
I20260807 19:49:11.007052 20245 catalog_manager.cc:4841] TS 8cb5e871046f461f917e4345fc3d89a1 (127.0.198.195:44077): tablet de20ad34f6824d22b77105e24962df12 (table test-table [id=6d31c5d73bfb4d0cb4fc697e1079b99c]) successfully deleted
I20260807 19:49:11.099216 20784 ts_tablet_manager.cc:1918] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:11.099526 20784 log.cc:1196] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/de20ad34f6824d22b77105e24962df12
I20260807 19:49:11.100148 20784 ts_tablet_manager.cc:1939] T de20ad34f6824d22b77105e24962df12 P 150d160fe3ce4471919692c8c50e78c7: Deleting consensus metadata
I20260807 19:49:11.100579 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20326
W20260807 19:49:11.122311 20244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:35375 (error 108)
I20260807 19:49:11.123183 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20466
I20260807 19:49:11.125308 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.123982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47892)
0807 19:49:11.124932 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.125089 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":698,"threads_started":1}
W20260807 19:49:11.125625 20244 catalog_manager.cc:4584] TS 150d160fe3ce4471919692c8c50e78c7 (127.0.198.193:35375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de20ad34f6824d22b77105e24962df12: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Aug 07 19:49:11 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:49:11.144819 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.124239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.0.198.194:37069)
0807 19:49:11.124508 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:11.124518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:11.124576 (+ 58us) server_negotiation.cc:407] Connection header received
0807 19:49:11.124675 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:11.124686 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:11.124799 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:11.124920 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:11.126790 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:11.128233 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:11.129774 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:11.130451 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:11.130588 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:11.136124 (+ 5536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:11.136149 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:11.136174 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:11.136271 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:11.140675 (+ 4404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:11.141831 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:11.141842 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:11.141846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:11.142024 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:11.142609 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:11.142615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:11.142621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:11.143211 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:11.143587 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:11.144134 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 19:49:11.144409 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 19:49:11.145725 20595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:11.179697 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.178971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47902)
0807 19:49:11.179280 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.179483 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 19:49:11.181191 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.180656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47914)
0807 19:49:11.180863 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.180971 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:49:11.240939 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20466
I20260807 19:49:11.243870 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.242955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47930)
0807 19:49:11.243551 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.243671 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435}
I20260807 19:49:11.245045 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.244659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47940)
0807 19:49:11.244820 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.244905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 19:49:11.266563 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20606
Aug 07 19:49:11 dist-test-slave-btw8 krb5kdc[20212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:49:11.288009 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.267822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.0.198.195:44077)
0807 19:49:11.268052 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:11.268059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:11.268093 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:49:11.268177 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:11.268184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:11.268283 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:11.268422 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:11.269935 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:11.271204 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:11.272863 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:11.273571 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:11.273733 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:11.279328 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:11.279343 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:11.279385 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:11.279451 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:11.284266 (+ 4815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:11.285313 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:11.285324 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:11.285327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:11.285539 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:11.286028 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:11.286034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:11.286037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:11.286601 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:11.286954 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:11.287295 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 19:49:11.287593 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:49:11.288951 20734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:11.325579 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.324978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47950)
0807 19:49:11.325282 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.325391 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 19:49:11.327003 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.326500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47964)
0807 19:49:11.326776 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.326854 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 19:49:11.420336 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20606
I20260807 19:49:11.443593 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20227
I20260807 19:49:11.445016 20294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:11.561585 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20227
I20260807 19:49:11.561712 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:11.560874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:35375 (local address 127.0.0.1:47968)
0807 19:49:11.561375 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:11.561497 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:35375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309}
2026-08-07T19:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5188 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T19:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:11Z Disabled control of system clock
I20260807 19:49:11.669513 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38179
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:38179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:12.051996 20796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:12.052349 20796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:12.052434 20796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:12.060410 20796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:12.060571 20796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:12.060613 20796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:12.060674 20796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:12.060712 20796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:12.070701 20796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:38179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:12.072232 20796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:12.074156 20796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:12.088424 20801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:12.088393 20802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:12.088804 20804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:12.089370 20796 server_base.cc:1034] running on GCE node
I20260807 19:49:12.090252 20796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:12.091993 20796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:12.093282 20796 hybrid_clock.cc:648] HybridClock initialized: now 1786132152093215 us; error 72 us; skew 500 ppm
I20260807 19:49:12.093701 20796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:12.096658 20796 webserver.cc:460] Webserver started at http://127.0.198.254:43153/ using document root <none> and password file <none>
I20260807 19:49:12.097560 20796 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:12.097730 20796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:12.098115 20796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:12.101799 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "ea26577293b7420c929ead5ca246a344"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "336483e5fed58d9ff550c0afe4b16edc"
server_key_iv: "aa1d5408a8168b7972f5f0df219cae6a"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.102850 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "ea26577293b7420c929ead5ca246a344"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "336483e5fed58d9ff550c0afe4b16edc"
server_key_iv: "aa1d5408a8168b7972f5f0df219cae6a"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.109012 20796 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:49:12.113276 20810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:12.114823 20796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:49:12.115074 20796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "ea26577293b7420c929ead5ca246a344"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "336483e5fed58d9ff550c0afe4b16edc"
server_key_iv: "aa1d5408a8168b7972f5f0df219cae6a"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.115437 20796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:12.129482 20796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:12.130445 20796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:12.130771 20796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:12.152508 20796 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:38179
I20260807 19:49:12.152529 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:38179 every 8 connection(s)
I20260807 19:49:12.154042 20796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:12.157581 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20796
I20260807 19:49:12.157769 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:12.158349 795 external_mini_cluster.cc:1428] Setting key 194ea9cfd4ffa7b5df7aea85ce9b44f6
I20260807 19:49:12.158877 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:12.174487 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: Bootstrap starting.
I20260807 19:49:12.175001 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:12.160563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.0.198.254:38179)
0807 19:49:12.161712 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:12.161724 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:12.161773 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:49:12.164170 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:12.164199 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:12.164804 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:12.165231 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:12.165995 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.167589 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.169760 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.170567 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.170757 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:12.172264 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:12.172275 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:12.172292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:12.172946 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:12.173034 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:12.173059 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:12.173176 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:12.173453 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:49:12.173864 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":402,"threads_started":1}
I20260807 19:49:12.177973 20864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:12.179059 20864 log.cc:824] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:12.182454 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: No bootstrap required, opened a new log
I20260807 19:49:12.190323 20864 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } }
I20260807 19:49:12.190793 20864 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:12.190902 20864 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea26577293b7420c929ead5ca246a344, State: Initialized, Role: FOLLOWER
I20260807 19:49:12.191651 20864 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } }
I20260807 19:49:12.191920 20864 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:12.192106 20864 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:12.192329 20864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:12.194943 20864 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } }
I20260807 19:49:12.195617 20864 leader_election.cc:302] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea26577293b7420c929ead5ca246a344; no voters:
I20260807 19:49:12.196357 20864 leader_election.cc:288] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:12.196585 20870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:12.197422 20870 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [term 1 LEADER]: Becoming Leader. State: Replica: ea26577293b7420c929ead5ca246a344, State: Running, Role: LEADER
I20260807 19:49:12.198071 20870 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } }
I20260807 19:49:12.199074 20864 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:12.201090 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea26577293b7420c929ead5ca246a344. Latest consensus state: current_term: 1 leader_uuid: "ea26577293b7420c929ead5ca246a344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } } }
I20260807 19:49:12.201352 20872 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea26577293b7420c929ead5ca246a344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea26577293b7420c929ead5ca246a344" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38179 } } }
I20260807 19:49:12.201584 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:12.201619 20872 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:12.204010 20878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:12.209466 20878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:12.220018 20878 catalog_manager.cc:1347] Generated new cluster ID: 66b14aa3646a42f8b376ee581e6a8dbb
I20260807 19:49:12.220161 20878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:12.240832 20878 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:12.242264 20878 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:12.255511 20878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: Generated new TSK 0
I20260807 19:49:12.256851 20878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:12.265455 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--builtin_ntp_servers=127.0.198.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:12.643671 20891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:12.643993 20891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:12.644085 20891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:12.651976 20891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:12.652119 20891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:12.652246 20891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:12.662317 20891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:12.663895 20891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:12.665839 20891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:12.680902 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:12.680948 20899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:12.681334 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:12.685498 20891 server_base.cc:1034] running on GCE node
I20260807 19:49:12.686300 20891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:12.687573 20891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:12.688846 20891 hybrid_clock.cc:648] HybridClock initialized: now 1786132152688792 us; error 48 us; skew 500 ppm
I20260807 19:49:12.689208 20891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:12.692193 20891 webserver.cc:460] Webserver started at http://127.0.198.193:34125/ using document root <none> and password file <none>
I20260807 19:49:12.692962 20891 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:12.693063 20891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:12.693373 20891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:12.696683 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "27a0a049a6b147c7b7d42f84e72503a9"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "940794b3d6af04dbd1ddc1457c46b9d9"
server_key_iv: "f259846d92962744a6c7d6854dbc6c7d"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.697563 20891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "27a0a049a6b147c7b7d42f84e72503a9"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "940794b3d6af04dbd1ddc1457c46b9d9"
server_key_iv: "f259846d92962744a6c7d6854dbc6c7d"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.703987 20891 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:49:12.708150 20905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:12.710112 20891 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:49:12.710307 20891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "27a0a049a6b147c7b7d42f84e72503a9"
format_stamp: "Formatted at 2026-08-07 19:49:12 on dist-test-slave-btw8"
server_key: "940794b3d6af04dbd1ddc1457c46b9d9"
server_key_iv: "f259846d92962744a6c7d6854dbc6c7d"
server_key_version: "encryptionkey@0"
I20260807 19:49:12.710582 20891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:12.741673 20891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:12.742759 20891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:12.743152 20891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:12.744524 20891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:12.746398 20891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:12.746656 20891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:12.746832 20891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:12.746946 20891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:12.784348 20891 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:39627
I20260807 19:49:12.784401 21020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:39627 every 8 connection(s)
I20260807 19:49:12.785928 20891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:12.794504 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 20891
I20260807 19:49:12.794685 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:12.795226 795 external_mini_cluster.cc:1428] Setting key be2dbe99fc852ef1fbf7eb6f566c93f3
I20260807 19:49:12.798260 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--builtin_ntp_servers=127.0.198.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:12.800463 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:12.788904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:41575 (local address 127.0.198.254:38179)
0807 19:49:12.789443 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:12.789448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:12.790872 (+ 1424us) server_negotiation.cc:407] Connection header received
0807 19:49:12.792903 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:12.792909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:12.793009 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:12.793149 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:12.794279 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.795158 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.797048 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.797485 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.797630 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:12.799289 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:12.799295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:12.799298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:12.799320 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:12.799427 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:12.799453 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:12.799594 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:12.799894 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 19:49:12.800049 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":298,"threads_started":1}
I20260807 19:49:12.801571 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:12.789619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:38179 (local address 127.0.198.193:41575)
0807 19:49:12.790690 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:12.790737 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:12.792598 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:12.793382 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:12.793390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:12.793492 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:12.794093 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:12.794110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.795356 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.795382 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:12.796892 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:12.796905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:12.797759 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:12.797769 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:12.797990 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:12.798866 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:12.798869 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:12.798928 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:12.799035 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:12.799595 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:12.799612 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:12.799624 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:49:12.799869 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 19:49:12.800341 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":306,"threads_started":1}
I20260807 19:49:12.803495 21021 heartbeater.cc:347] Connected to a master server at 127.0.198.254:38179
I20260807 19:49:12.803844 21021 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:12.804710 21021 heartbeater.cc:511] Master 127.0.198.254:38179 requested a full tablet report, sending...
I20260807 19:49:12.807292 20828 ts_manager.cc:194] Registered new tserver with Master: 27a0a049a6b147c7b7d42f84e72503a9 (127.0.198.193:39627)
I20260807 19:49:12.809840 20828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.193:41575
W20260807 19:49:13.183825 21025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:13.184157 21025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:13.184248 21025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:13.191994 21025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:13.192121 21025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:13.192229 21025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:13.202301 21025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:13.203694 21025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:13.205487 21025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:13.221324 21030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:13.221349 21031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:13.221261 21033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:13.225528 21025 server_base.cc:1034] running on GCE node
I20260807 19:49:13.226460 21025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:13.227761 21025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:13.229065 21025 hybrid_clock.cc:648] HybridClock initialized: now 1786132153228946 us; error 111 us; skew 500 ppm
I20260807 19:49:13.229493 21025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:13.232928 21025 webserver.cc:460] Webserver started at http://127.0.198.194:38849/ using document root <none> and password file <none>
I20260807 19:49:13.233822 21025 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:13.233937 21025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:13.234375 21025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:13.238027 21025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "1b2e2d5e1e8843bf8307d4757427ff4c"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "36b82cb545423086b58b422d01193524"
server_key_iv: "55c6c707c0a392cdadd59ce236d1a6cf"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.239171 21025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "1b2e2d5e1e8843bf8307d4757427ff4c"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "36b82cb545423086b58b422d01193524"
server_key_iv: "55c6c707c0a392cdadd59ce236d1a6cf"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.245713 21025 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:13.249874 21039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.251677 21025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:49:13.251950 21025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "1b2e2d5e1e8843bf8307d4757427ff4c"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "36b82cb545423086b58b422d01193524"
server_key_iv: "55c6c707c0a392cdadd59ce236d1a6cf"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.252218 21025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:13.276574 21025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:13.277556 21025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:13.277918 21025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:13.279048 21025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:13.280886 21025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:13.280982 21025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.281127 21025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:13.281237 21025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.318414 21025 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:35247
I20260807 19:49:13.318531 21156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:35247 every 8 connection(s)
I20260807 19:49:13.320258 21025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:13.320978 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21025
I20260807 19:49:13.321188 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:13.321868 795 external_mini_cluster.cc:1428] Setting key 1c92069f6f681aac9fa168072b331f0e
I20260807 19:49:13.326851 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--builtin_ntp_servers=127.0.198.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:13.336264 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.323709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:55839 (local address 127.0.198.254:38179)
0807 19:49:13.324209 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.324215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.325553 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 19:49:13.327821 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.327829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.327930 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.328181 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:13.329568 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.330417 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.332644 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.333141 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.333345 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.335171 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:13.335177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:13.335181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:13.335207 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:13.335254 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:13.335269 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:13.335403 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.335739 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.335869 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260807 19:49:13.337548 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.324413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:38179 (local address 127.0.198.194:55839)
0807 19:49:13.325350 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:13.325397 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:13.327475 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:13.328472 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:13.328483 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:13.328597 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:13.329345 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.329367 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.330629 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.330633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.332420 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.332439 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.333479 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.333492 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.333780 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.334711 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:13.334714 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:13.334761 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:13.334863 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:13.335418 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:13.335433 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:13.335448 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 19:49:13.335732 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 19:49:13.336229 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":863,"threads_started":1}
I20260807 19:49:13.339475 21158 heartbeater.cc:347] Connected to a master server at 127.0.198.254:38179
I20260807 19:49:13.339886 21158 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:13.340797 21158 heartbeater.cc:511] Master 127.0.198.254:38179 requested a full tablet report, sending...
I20260807 19:49:13.342562 20828 ts_manager.cc:194] Registered new tserver with Master: 1b2e2d5e1e8843bf8307d4757427ff4c (127.0.198.194:35247)
I20260807 19:49:13.343575 20828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.194:55839
W20260807 19:49:13.711113 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:13.711480 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:13.711678 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:13.719748 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:13.719874 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:13.720038 21162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:13.730027 21162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:38179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:13.731565 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:13.733945 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:13.749912 21168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:13.749890 21170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:13.750280 21167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:13.750761 21162 server_base.cc:1034] running on GCE node
I20260807 19:49:13.751609 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:13.752684 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:13.753908 21162 hybrid_clock.cc:648] HybridClock initialized: now 1786132153753858 us; error 41 us; skew 500 ppm
I20260807 19:49:13.754299 21162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:13.767331 21162 webserver.cc:460] Webserver started at http://127.0.198.195:36629/ using document root <none> and password file <none>
I20260807 19:49:13.768221 21162 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:13.768329 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:13.768616 21162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:13.771842 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "f784635116a545ff861f98a06d6880d9"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "77062123d4984c32f069e98a8ad76cce"
server_key_iv: "bf1ec549f8c3dbf3196227a9d28d8c5d"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.772737 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "f784635116a545ff861f98a06d6880d9"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "77062123d4984c32f069e98a8ad76cce"
server_key_iv: "bf1ec549f8c3dbf3196227a9d28d8c5d"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.778813 21162 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.006s
I20260807 19:49:13.782723 21176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.784303 21162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:49:13.784509 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "f784635116a545ff861f98a06d6880d9"
format_stamp: "Formatted at 2026-08-07 19:49:13 on dist-test-slave-btw8"
server_key: "77062123d4984c32f069e98a8ad76cce"
server_key_iv: "bf1ec549f8c3dbf3196227a9d28d8c5d"
server_key_version: "encryptionkey@0"
I20260807 19:49:13.784786 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:13.807453 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:13.808422 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:13.808768 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:13.809911 21162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:13.811656 21162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:13.811750 21162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.811914 21162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:13.812008 21162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:13.814512 21021 heartbeater.cc:503] Master 127.0.198.254:38179 was elected leader, sending a full tablet report...
I20260807 19:49:13.852056 21162 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:36129
I20260807 19:49:13.852166 21294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:36129 every 8 connection(s)
I20260807 19:49:13.853451 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:13.856534 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21162
I20260807 19:49:13.856737 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:13.857352 795 external_mini_cluster.cc:1428] Setting key 5d2c0b09feb26618da43c3a0a0fd46e4
I20260807 19:49:13.869691 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.856369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:45011 (local address 127.0.198.254:38179)
0807 19:49:13.857216 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.857222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.858640 (+ 1418us) server_negotiation.cc:407] Connection header received
0807 19:49:13.861185 (+ 2545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.861192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.861316 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.861473 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:13.862848 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.863879 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.865869 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.866377 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.866570 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.868505 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:13.868511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:13.868515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:13.868543 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:13.868594 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:13.868611 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:13.868740 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.869052 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.869187 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":578,"threads_started":1}
I20260807 19:49:13.870918 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.857099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:38179 (local address 127.0.198.195:45011)
0807 19:49:13.858438 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:13.858483 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:13.860796 (+ 2313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:13.861786 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:13.861797 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:13.861907 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:13.862623 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.862643 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.864085 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.864093 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.865655 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.865672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.866732 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.866745 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.867065 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.868066 (+ 1001us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:13.868070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:13.868119 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:13.868229 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:13.868742 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:13.868759 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:13.868772 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 19:49:13.869025 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 19:49:13.869551 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":449,"threads_started":1}
I20260807 19:49:13.872819 21295 heartbeater.cc:347] Connected to a master server at 127.0.198.254:38179
I20260807 19:49:13.873215 21295 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:13.874250 21295 heartbeater.cc:511] Master 127.0.198.254:38179 requested a full tablet report, sending...
I20260807 19:49:13.875836 20828 ts_manager.cc:194] Registered new tserver with Master: f784635116a545ff861f98a06d6880d9 (127.0.198.195:36129)
I20260807 19:49:13.876755 20828 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.195:45011
I20260807 19:49:13.887405 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:13.898602 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.892192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.0.198.254:38179)
0807 19:49:13.892433 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.892443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.892799 (+ 356us) server_negotiation.cc:407] Connection header received
0807 19:49:13.892951 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.892955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.893024 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.893115 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:13.893692 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.894459 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.896000 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.896472 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.896582 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.897796 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:13.897801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:13.897804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:13.897824 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:13.897860 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:13.897873 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:13.898002 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.898177 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.898273 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:49:13.904457 20828 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:13.913781 20828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:13.958192 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.930268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.0.198.194:35247)
0807 19:49:13.936666 (+ 6398us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.936682 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.946523 (+ 9841us) server_negotiation.cc:407] Connection header received
0807 19:49:13.946672 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.946684 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.946982 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.947253 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:13.949255 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950585 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.954765 (+ 4180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.956080 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956237 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.956496 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.957584 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.957867 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6181,"thread_start_us":6029,"threads_started":1}
I20260807 19:49:13.958495 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.928937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.0.198.195:36129)
0807 19:49:13.946713 (+ 17776us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.946731 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.946762 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:49:13.946853 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.946860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.947042 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.947251 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:13.948494 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950441 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.954357 (+ 3916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.955904 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956041 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.956367 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.957920 (+ 1553us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.958116 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17557,"thread_start_us":17360,"threads_started":1}
I20260807 19:49:13.958622 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.930400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.0.198.193:39627)
0807 19:49:13.938499 (+ 8099us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:13.938510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:13.947153 (+ 8643us) server_negotiation.cc:407] Connection header received
0807 19:49:13.947384 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:13.947393 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:13.947663 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:13.947888 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:13.948860 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950653 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.954008 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.955809 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956016 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.956368 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:13.957611 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0807 19:49:13.957883 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7901,"thread_start_us":7728,"threads_started":1}
I20260807 19:49:13.959232 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.929300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35247 (local address 127.0.0.1:53854)
0807 19:49:13.946336 (+ 17036us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:13.946367 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:13.946518 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:13.947578 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:13.947584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:13.947649 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:13.948881 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.948898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950792 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.950801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.954551 (+ 3750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.954558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.956753 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.957076 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.957294 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 19:49:13.958472 (+ 1178us) client_negotiation.cc:240] Negotiation successful
0807 19:49:13.958768 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15571,"mutex_wait_us":2110,"thread_start_us":15734,"threads_started":1}
I20260807 19:49:13.959195 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.928746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:36129 (local address 127.0.0.1:58474)
0807 19:49:13.945199 (+ 16453us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:13.945305 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:13.945695 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:13.947392 (+ 1697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:13.947401 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:13.947435 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:13.948197 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.948215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950652 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.950660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.954111 (+ 3451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.954122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.956578 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956587 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.957304 (+ 717us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.957340 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:49:13.957442 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:49:13.957702 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15868,"mutex_wait_us":669,"thread_start_us":18759,"threads_started":1}
I20260807 19:49:13.959545 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:13.929565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:39627 (local address 127.0.0.1:45486)
0807 19:49:13.946966 (+ 17401us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:13.946987 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:13.947163 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:13.947941 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:13.947946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:13.947966 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:13.948601 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.948622 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.950828 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.950836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.953756 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:13.953777 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:13.956687 (+ 2910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:13.956695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:13.957102 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:13.957437 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 19:49:13.958885 (+ 1448us) client_negotiation.cc:240] Negotiation successful
0807 19:49:13.959039 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17284,"mutex_wait_us":636,"thread_start_us":16402,"threads_started":1}
I20260807 19:49:13.963410 21228 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9c7ac456c847daa56ac620a60641bb (DEFAULT_TABLE table=test-table [id=ad32f169695c413bb5dce37e44a33a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:13.963279 21090 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9c7ac456c847daa56ac620a60641bb (DEFAULT_TABLE table=test-table [id=ad32f169695c413bb5dce37e44a33a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:13.963419 20954 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9c7ac456c847daa56ac620a60641bb (DEFAULT_TABLE table=test-table [id=ad32f169695c413bb5dce37e44a33a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:13.965250 21228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9c7ac456c847daa56ac620a60641bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:13.965359 20954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9c7ac456c847daa56ac620a60641bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:13.965621 21090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a9c7ac456c847daa56ac620a60641bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:13.980576 21314 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Bootstrap starting.
I20260807 19:49:13.984387 21315 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Bootstrap starting.
I20260807 19:49:13.986951 21314 tablet_bootstrap.cc:654] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:13.988308 21316 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Bootstrap starting.
I20260807 19:49:13.988618 21315 tablet_bootstrap.cc:654] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:13.988621 21314 log.cc:824] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:13.990218 21315 log.cc:824] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:13.992348 21316 tablet_bootstrap.cc:654] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:13.993440 21316 log.cc:824] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:13.993575 21314 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: No bootstrap required, opened a new log
I20260807 19:49:13.993964 21314 ts_tablet_manager.cc:1397] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260807 19:49:14.002600 21316 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: No bootstrap required, opened a new log
I20260807 19:49:14.002967 21316 ts_tablet_manager.cc:1397] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260807 19:49:14.003541 21315 tablet_bootstrap.cc:492] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: No bootstrap required, opened a new log
I20260807 19:49:14.003851 21315 ts_tablet_manager.cc:1397] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.015s
I20260807 19:49:14.008244 21314 raft_consensus.cc:348] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.008795 21314 raft_consensus.cc:374] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:14.008913 21314 raft_consensus.cc:729] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2e2d5e1e8843bf8307d4757427ff4c, State: Initialized, Role: FOLLOWER
I20260807 19:49:14.009743 21314 consensus_queue.cc:260] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.011117 21158 heartbeater.cc:503] Master 127.0.198.254:38179 was elected leader, sending a full tablet report...
I20260807 19:49:14.011909 21314 ts_tablet_manager.cc:1428] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260807 19:49:14.014118 21316 raft_consensus.cc:348] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.014763 21316 raft_consensus.cc:374] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:14.014519 21315 raft_consensus.cc:348] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.014917 21316 raft_consensus.cc:729] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f784635116a545ff861f98a06d6880d9, State: Initialized, Role: FOLLOWER
I20260807 19:49:14.015007 21315 raft_consensus.cc:374] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:14.015105 21315 raft_consensus.cc:729] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a0a049a6b147c7b7d42f84e72503a9, State: Initialized, Role: FOLLOWER
I20260807 19:49:14.015841 21315 consensus_queue.cc:260] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.015933 21316 consensus_queue.cc:260] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.017946 21315 ts_tablet_manager.cc:1428] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260807 19:49:14.018950 21295 heartbeater.cc:503] Master 127.0.198.254:38179 was elected leader, sending a full tablet report...
I20260807 19:49:14.020815 21316 ts_tablet_manager.cc:1428] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Time spent starting tablet: real 0.018s user 0.007s sys 0.009s
W20260807 19:49:14.038956 21022 tablet.cc:2369] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:49:14.075305 21159 tablet.cc:2369] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:14.079298 21321 raft_consensus.cc:482] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.079748 21321 raft_consensus.cc:504] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.081652 21321 leader_election.cc:288] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f784635116a545ff861f98a06d6880d9 (127.0.198.195:36129), 1b2e2d5e1e8843bf8307d4757427ff4c (127.0.198.194:35247)
I20260807 19:49:14.090773 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.082233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:39425 (local address 127.0.198.194:35247)
0807 19:49:14.082446 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.082453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.083611 (+ 1158us) server_negotiation.cc:407] Connection header received
0807 19:49:14.083816 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.083821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.083894 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.084000 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:14.084624 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.085354 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.088169 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.089468 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.089672 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.089759 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.090353 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.090497 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:49:14.090817 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.082295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:51477 (local address 127.0.198.195:36129)
0807 19:49:14.082538 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.082547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.083257 (+ 710us) server_negotiation.cc:407] Connection header received
0807 19:49:14.083562 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.083566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.083629 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.083734 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:14.084325 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.085041 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.088168 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.089558 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.089737 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.089810 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.090353 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.090497 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:49:14.091045 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.082632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:35247 (local address 127.0.198.193:39425)
0807 19:49:14.083496 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:14.083512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:14.083677 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:14.084034 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:14.084038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:14.084066 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:14.084448 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:14.084457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.085504 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.085509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:14.087875 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:14.087894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.089844 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.089850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:14.090060 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.090178 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 19:49:14.090285 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:49:14.090575 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":182,"thread_start_us":519,"threads_started":1}
I20260807 19:49:14.091087 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.082118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:36129 (local address 127.0.198.193:51477)
0807 19:49:14.083130 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:14.083148 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:14.083395 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:14.083757 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:14.083762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:14.083788 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:14.084166 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:14.084177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.085194 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.085202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:14.087875 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:14.087894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.089831 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.089839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:14.090116 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.090182 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 19:49:14.090286 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:49:14.090579 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":218,"thread_start_us":627,"threads_started":1}
I20260807 19:49:14.092164 21249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9c7ac456c847daa56ac620a60641bb" candidate_uuid: "27a0a049a6b147c7b7d42f84e72503a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f784635116a545ff861f98a06d6880d9" is_pre_election: true
I20260807 19:49:14.092163 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9c7ac456c847daa56ac620a60641bb" candidate_uuid: "27a0a049a6b147c7b7d42f84e72503a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" is_pre_election: true
I20260807 19:49:14.092713 21249 raft_consensus.cc:2454] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a0a049a6b147c7b7d42f84e72503a9 in term 0.
I20260807 19:49:14.092717 21110 raft_consensus.cc:2454] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a0a049a6b147c7b7d42f84e72503a9 in term 0.
I20260807 19:49:14.093436 20909 leader_election.cc:302] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b2e2d5e1e8843bf8307d4757427ff4c, 27a0a049a6b147c7b7d42f84e72503a9, f784635116a545ff861f98a06d6880d9; no voters:
I20260807 19:49:14.093931 21321 raft_consensus.cc:2781] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:14.094213 21321 raft_consensus.cc:482] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.094317 21321 raft_consensus.cc:3037] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:14.096696 21321 raft_consensus.cc:504] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
I20260807 19:49:14.097355 21321 leader_election.cc:288] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [CANDIDATE]: Term 1 election: Requested vote from peers f784635116a545ff861f98a06d6880d9 (127.0.198.195:36129), 1b2e2d5e1e8843bf8307d4757427ff4c (127.0.198.194:35247)
I20260807 19:49:14.097939 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9c7ac456c847daa56ac620a60641bb" candidate_uuid: "27a0a049a6b147c7b7d42f84e72503a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c"
I20260807 19:49:14.097946 21249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a9c7ac456c847daa56ac620a60641bb" candidate_uuid: "27a0a049a6b147c7b7d42f84e72503a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f784635116a545ff861f98a06d6880d9"
I20260807 19:49:14.098155 21110 raft_consensus.cc:3037] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:14.098192 21249 raft_consensus.cc:3037] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:14.100755 21110 raft_consensus.cc:2454] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a0a049a6b147c7b7d42f84e72503a9 in term 1.
I20260807 19:49:14.101120 21249 raft_consensus.cc:2454] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a0a049a6b147c7b7d42f84e72503a9 in term 1.
I20260807 19:49:14.101364 20907 leader_election.cc:302] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2e2d5e1e8843bf8307d4757427ff4c, 27a0a049a6b147c7b7d42f84e72503a9; no voters:
I20260807 19:49:14.101794 21321 raft_consensus.cc:2781] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:14.102330 21321 raft_consensus.cc:686] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 1 LEADER]: Becoming Leader. State: Replica: 27a0a049a6b147c7b7d42f84e72503a9, State: Running, Role: LEADER
I20260807 19:49:14.102919 21321 consensus_queue.cc:237] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } }
W20260807 19:49:14.105310 21296 tablet.cc:2369] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:14.108237 20827 catalog_manager.cc:5495] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a0a049a6b147c7b7d42f84e72503a9 (127.0.198.193). New cstate: current_term: 1 leader_uuid: "27a0a049a6b147c7b7d42f84e72503a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a0a049a6b147c7b7d42f84e72503a9" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.126336 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.120349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.0.198.193:39627)
0807 19:49:14.120545 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.120551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.120569 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:49:14.120668 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.120672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.120813 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.120963 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:14.121387 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.122218 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.123801 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.124236 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.124348 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.125005 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:14.125686 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:14.125850 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.126071 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.126120 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:49:14.135764 21249 raft_consensus.cc:1261] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 1 FOLLOWER]: Refusing update from remote peer 27a0a049a6b147c7b7d42f84e72503a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:14.135764 21110 raft_consensus.cc:1261] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 1 FOLLOWER]: Refusing update from remote peer 27a0a049a6b147c7b7d42f84e72503a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:14.136992 21321 consensus_queue.cc:1035] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b2e2d5e1e8843bf8307d4757427ff4c" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:14.137370 21325 consensus_queue.cc:1035] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f784635116a545ff861f98a06d6880d9" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 36129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:14.172381 20827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:14.172827 20827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:14.182235 20827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea26577293b7420c929ead5ca246a344: Sending DeleteTablet for 3 replicas of tablet 2a9c7ac456c847daa56ac620a60641bb
I20260807 19:49:14.184300 20954 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9c7ac456c847daa56ac620a60641bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:14 UTC) from {username='slave'} at 127.0.0.1:45486
I20260807 19:49:14.186018 21228 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9c7ac456c847daa56ac620a60641bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:14 UTC) from {username='slave'} at 127.0.0.1:58474
I20260807 19:49:14.188503 21090 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9c7ac456c847daa56ac620a60641bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:14 UTC) from {username='slave'} at 127.0.0.1:53854
I20260807 19:49:14.192132 20953 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9c7ac456c847daa56ac620a60641bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:14 UTC) from {username='slave'} at 127.0.0.1:45486
I20260807 19:49:14.198137 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20891
I20260807 19:49:14.203140 21339 tablet_replica.cc:330] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: stopping tablet replica
I20260807 19:49:14.203716 21339 raft_consensus.cc:2229] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:14.204317 21339 raft_consensus.cc:2258] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:14.206938 21339 ts_tablet_manager.cc:1905] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:14.224287 21339 ts_tablet_manager.cc:1918] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:14.224537 21339 log.cc:1196] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/2a9c7ac456c847daa56ac620a60641bb
I20260807 19:49:14.225235 21339 ts_tablet_manager.cc:1939] T 2a9c7ac456c847daa56ac620a60641bb P 27a0a049a6b147c7b7d42f84e72503a9: Deleting consensus metadata
I20260807 19:49:14.227584 20811 catalog_manager.cc:4841] TS 27a0a049a6b147c7b7d42f84e72503a9 (127.0.198.193:39627): tablet 2a9c7ac456c847daa56ac620a60641bb (table test-table [id=ad32f169695c413bb5dce37e44a33a03]) successfully deleted
W20260807 19:49:14.231871 20811 catalog_manager.cc:4808] TS 27a0a049a6b147c7b7d42f84e72503a9 (127.0.198.193:39627): delete failed for tablet 2a9c7ac456c847daa56ac620a60641bb because the tablet was not found. No further retry: Not found: Tablet not found: 2a9c7ac456c847daa56ac620a60641bb
I20260807 19:49:14.241348 21340 tablet_replica.cc:330] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: stopping tablet replica
I20260807 19:49:14.251550 21341 tablet_replica.cc:330] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: stopping tablet replica
I20260807 19:49:14.259227 21341 raft_consensus.cc:2229] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:14.260581 21341 raft_consensus.cc:2258] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:14.261984 21340 raft_consensus.cc:2229] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:14.262961 21340 raft_consensus.cc:2258] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:14.266870 21340 ts_tablet_manager.cc:1905] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:14.269879 21341 ts_tablet_manager.cc:1905] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:14.274788 21340 ts_tablet_manager.cc:1918] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:14.275012 21340 log.cc:1196] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/2a9c7ac456c847daa56ac620a60641bb
I20260807 19:49:14.275638 21340 ts_tablet_manager.cc:1939] T 2a9c7ac456c847daa56ac620a60641bb P f784635116a545ff861f98a06d6880d9: Deleting consensus metadata
I20260807 19:49:14.278379 21341 ts_tablet_manager.cc:1918] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:14.278419 20814 catalog_manager.cc:4841] TS f784635116a545ff861f98a06d6880d9 (127.0.198.195:36129): tablet 2a9c7ac456c847daa56ac620a60641bb (table test-table [id=ad32f169695c413bb5dce37e44a33a03]) successfully deleted
I20260807 19:49:14.278724 21341 log.cc:1196] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/2a9c7ac456c847daa56ac620a60641bb
I20260807 19:49:14.279327 21341 ts_tablet_manager.cc:1939] T 2a9c7ac456c847daa56ac620a60641bb P 1b2e2d5e1e8843bf8307d4757427ff4c: Deleting consensus metadata
I20260807 19:49:14.281219 20812 catalog_manager.cc:4841] TS 1b2e2d5e1e8843bf8307d4757427ff4c (127.0.198.194:35247): tablet 2a9c7ac456c847daa56ac620a60641bb (table test-table [id=ad32f169695c413bb5dce37e44a33a03]) successfully deleted
I20260807 19:49:14.284195 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.200702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.0.198.193:39627)
0807 19:49:14.200918 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.200926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.243552 (+ 42626us) server_negotiation.cc:407] Connection header received
0807 19:49:14.244006 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.244018 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.244278 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.244400 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:14.245467 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.247339 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.279571 (+ 32232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.280414 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.280560 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.282473 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:14.282491 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:14.282518 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:14.283175 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:14.283230 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:14.283260 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:14.283462 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.283627 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.283797 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:49:14.284976 21016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:14.394610 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20891
I20260807 19:49:14.417594 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21025
I20260807 19:49:14.425002 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.418390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.0.198.194:35247)
0807 19:49:14.418584 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.418592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.418669 (+ 77us) server_negotiation.cc:407] Connection header received
0807 19:49:14.418952 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.418957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.419060 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.419179 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:14.419642 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.420436 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.421757 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.422170 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.422313 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.423555 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:14.423564 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:14.423579 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:14.424231 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:14.424292 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:14.424314 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:14.424443 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.424552 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.424665 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:49:14.425858 21152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:14.522420 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21025
I20260807 19:49:14.543717 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21162
I20260807 19:49:14.552003 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:14.544820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.0.198.195:36129)
0807 19:49:14.545029 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:14.545035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:14.545059 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:49:14.545273 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:14.545278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:14.545379 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:14.545497 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:14.545969 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.546827 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.548467 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:14.548902 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:14.549011 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:14.550300 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:14.550309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:14.550324 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:14.550949 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:14.551001 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:14.551023 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:14.551127 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:14.551467 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 19:49:14.551577 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:49:14.552768 21290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:14.647514 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21162
I20260807 19:49:14.666975 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 20796
I20260807 19:49:14.668380 20859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:14.786242 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 20796
2026-08-07T19:49:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:14 dist-test-slave-btw8 krb5kdc[21350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:14 dist-test-slave-btw8 krb5kdc[21350](info): set up 2 sockets
Aug 07 19:49:14 dist-test-slave-btw8 krb5kdc[21350](info): commencing operation
krb5kdc: starting...
W20260807 19:49:16.062523 795 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:16 dist-test-slave-btw8 krb5kdc[21350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:49:16.435708 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37471
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:16.806700 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:16.807054 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:16.807220 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:16.814980 21365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:16.815125 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:16.815182 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:16.815316 21365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:16.815402 21365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:16.826017 21365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:16.827633 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:16.829097 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:16.843452 21370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:16.843583 21373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:16.843858 21371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:16.845182 21365 server_base.cc:1034] running on GCE node
I20260807 19:49:16.846356 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:16.848428 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:16.849774 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786132156849724 us; error 51 us; skew 500 ppm
I20260807 19:49:16.850281 21365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:16 dist-test-slave-btw8 krb5kdc[21350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:16.854557 21365 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:16.856694 21365 webserver.cc:460] Webserver started at http://127.0.198.254:32889/ using document root <none> and password file <none>
I20260807 19:49:16.857589 21365 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:16.857780 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:16.858181 21365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:16.861917 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "f43301e5bf544e9a9126843c630ad86a"
format_stamp: "Formatted at 2026-08-07 19:49:16 on dist-test-slave-btw8"
server_key: "4dcf138b7538a0f320cb85c4eb5ca4e4"
server_key_iv: "9ed24dc9510a87505612cf7908f69b20"
server_key_version: "encryptionkey@0"
I20260807 19:49:16.862939 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "f43301e5bf544e9a9126843c630ad86a"
format_stamp: "Formatted at 2026-08-07 19:49:16 on dist-test-slave-btw8"
server_key: "4dcf138b7538a0f320cb85c4eb5ca4e4"
server_key_iv: "9ed24dc9510a87505612cf7908f69b20"
server_key_version: "encryptionkey@0"
I20260807 19:49:16.869925 21365 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:16.874758 21382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:16.876698 21365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:16.876992 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "f43301e5bf544e9a9126843c630ad86a"
format_stamp: "Formatted at 2026-08-07 19:49:16 on dist-test-slave-btw8"
server_key: "4dcf138b7538a0f320cb85c4eb5ca4e4"
server_key_iv: "9ed24dc9510a87505612cf7908f69b20"
server_key_version: "encryptionkey@0"
I20260807 19:49:16.877279 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:16.925541 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:16.930838 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:16.931231 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:16.954803 21365 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:37471
I20260807 19:49:16.954845 21434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:37471 every 8 connection(s)
I20260807 19:49:16.956483 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:16.961136 21435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:16.966257 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21365
I20260807 19:49:16.966531 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:16.967031 795 external_mini_cluster.cc:1428] Setting key 67e539a15f128ad90ae1afeec1768ece
I20260807 19:49:16.974910 21435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: Bootstrap starting.
I20260807 19:49:16.979178 21435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:16.980651 21435 log.cc:824] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:16.984298 21435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: No bootstrap required, opened a new log
Aug 07 19:49:16 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:16.992956 21435 raft_consensus.cc:348] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } }
I20260807 19:49:16.993476 21435 raft_consensus.cc:374] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:16.993634 21435 raft_consensus.cc:729] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43301e5bf544e9a9126843c630ad86a, State: Initialized, Role: FOLLOWER
I20260807 19:49:16.994380 21435 consensus_queue.cc:260] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } }
I20260807 19:49:16.994701 21435 raft_consensus.cc:388] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:16.994874 21435 raft_consensus.cc:482] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:16.995085 21435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:16.995179 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:16.969226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.0.198.254:37471)
0807 19:49:16.970201 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:16.970214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:16.970270 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:49:16.973126 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:16.973159 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:16.973701 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:16.974107 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:16.976321 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:16.977881 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:16.980073 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:16.981013 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:16.981195 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:16.986058 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:16.986067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:16.986084 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:16.986132 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:16.989871 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:16.990881 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:16.990893 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:16.990901 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:16.991123 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:16.991645 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:16.991649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:16.991651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:16.992342 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:16.992773 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:16.993174 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:49:16.993874 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":479,"threads_started":1}
I20260807 19:49:16.998322 21435 raft_consensus.cc:504] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } }
I20260807 19:49:16.999051 21435 leader_election.cc:302] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f43301e5bf544e9a9126843c630ad86a; no voters:
I20260807 19:49:16.999779 21435 leader_election.cc:288] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:16.999954 21441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:17.002040 21441 raft_consensus.cc:686] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [term 1 LEADER]: Becoming Leader. State: Replica: f43301e5bf544e9a9126843c630ad86a, State: Running, Role: LEADER
I20260807 19:49:17.002632 21441 consensus_queue.cc:237] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } }
I20260807 19:49:17.003023 21435 sys_catalog.cc:564] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:17.005899 21442 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f43301e5bf544e9a9126843c630ad86a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } } }
I20260807 19:49:17.006376 21442 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:17.005843 21443 sys_catalog.cc:455] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f43301e5bf544e9a9126843c630ad86a. Latest consensus state: current_term: 1 leader_uuid: "f43301e5bf544e9a9126843c630ad86a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f43301e5bf544e9a9126843c630ad86a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37471 } } }
I20260807 19:49:17.006938 21443 sys_catalog.cc:458] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:17.017086 21448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:17.022053 21448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:17.033078 21448 catalog_manager.cc:1347] Generated new cluster ID: 3a37aa6280a34b2bb396d82d07429c99
I20260807 19:49:17.033237 21448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:17.053756 21448 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:17.055887 21448 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:17.071259 21448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: Generated new TSK 0
I20260807 19:49:17.072602 21448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.198.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
Entry for principal kudu/127.0.198.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab.
I20260807 19:49:17.167711 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:17.557615 21462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:17.558020 21462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:17.558208 21462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:17.565846 21462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:17.566044 21462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:17.566247 21462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:17.577576 21462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.193.keytab
--principal=kudu/127.0.198.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:17.579237 21462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:17.580830 21462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:17.597136 21467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:17.597160 21468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:17.597457 21470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:17.598177 21462 server_base.cc:1034] running on GCE node
I20260807 19:49:17.598809 21462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:17.599918 21462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:17.601161 21462 hybrid_clock.cc:648] HybridClock initialized: now 1786132157601103 us; error 49 us; skew 500 ppm
I20260807 19:49:17.601522 21462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:17 dist-test-slave-btw8 krb5kdc[21350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:17.605402 21462 init.cc:377] Logged in from keytab as kudu/127.0.198.193@KRBTEST.COM (short username kudu)
I20260807 19:49:17.607023 21462 webserver.cc:460] Webserver started at http://127.0.198.193:43609/ using document root <none> and password file <none>
I20260807 19:49:17.607842 21462 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:17.607990 21462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:17.608407 21462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:17.612025 21462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "b161dd3834224094bc7242f94299706c"
format_stamp: "Formatted at 2026-08-07 19:49:17 on dist-test-slave-btw8"
server_key: "3c34de852b4994c0c63dba1f4e8a43af"
server_key_iv: "8e358a3145022eb024f0c381beba542f"
server_key_version: "encryptionkey@0"
I20260807 19:49:17.612941 21462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "b161dd3834224094bc7242f94299706c"
format_stamp: "Formatted at 2026-08-07 19:49:17 on dist-test-slave-btw8"
server_key: "3c34de852b4994c0c63dba1f4e8a43af"
server_key_iv: "8e358a3145022eb024f0c381beba542f"
server_key_version: "encryptionkey@0"
I20260807 19:49:17.618832 21462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:49:17.623332 21478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:17.625134 21462 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:17.625387 21462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "b161dd3834224094bc7242f94299706c"
format_stamp: "Formatted at 2026-08-07 19:49:17 on dist-test-slave-btw8"
server_key: "3c34de852b4994c0c63dba1f4e8a43af"
server_key_iv: "8e358a3145022eb024f0c381beba542f"
server_key_version: "encryptionkey@0"
I20260807 19:49:17.625742 21462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:17.639554 21462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:17.644465 21462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:17.644847 21462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:17.646020 21462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:17.647588 21462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:17.647717 21462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:17.647877 21462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:17.647954 21462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:17.687662 21462 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:42323
I20260807 19:49:17.687732 21594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:42323 every 8 connection(s)
I20260807 19:49:17.689077 21462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:17.699541 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21462
I20260807 19:49:17.699764 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:17.700253 795 external_mini_cluster.cc:1428] Setting key 161ef4af0163beeaec17903564a06985
Aug 07 19:49:17 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.193@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:17.712750 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:17.692096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:43597 (local address 127.0.198.254:37471)
0807 19:49:17.692587 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:17.692592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:17.693819 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 19:49:17.696014 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:17.696020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:17.696136 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:17.696274 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:17.698087 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:17.698839 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:17.700813 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:17.701260 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:17.701433 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:17.706075 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:17.706081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:17.706084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:17.706122 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:17.709224 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:17.710166 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:17.710172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:17.710180 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:17.710317 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:17.710818 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:17.710824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:17.710827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:17.711112 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:17.711298 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:17.712117 (+ 819us) server_negotiation.cc:299] Negotiation successful
0807 19:49:17.712377 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":243,"threads_started":1}
I20260807 19:49:17.713821 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:17.692719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37471 (local address 127.0.198.193:43597)
0807 19:49:17.693534 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:17.693669 (+ 135us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:17.695662 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:17.696562 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:17.696572 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:17.697200 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:17.697853 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:17.697874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:17.699047 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:17.699051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:17.700640 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:17.700661 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:17.701524 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:17.701530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:17.701753 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:17.702658 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:17.702696 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:17.705779 (+ 3083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:17.709408 (+ 3629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:17.709419 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:17.709439 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:17.710011 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:17.710441 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:17.710446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:17.710449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:17.710686 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:17.711260 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:17.711269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:17.711706 (+ 437us) client_negotiation.cc:769] Sending connection context
0807 19:49:17.711994 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 19:49:17.712579 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":354,"threads_started":1}
I20260807 19:49:17.715700 21595 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37471
I20260807 19:49:17.716104 21595 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:17.717135 21595 heartbeater.cc:511] Master 127.0.198.254:37471 requested a full tablet report, sending...
I20260807 19:49:17.720105 21400 ts_manager.cc:194] Registered new tserver with Master: b161dd3834224094bc7242f94299706c (127.0.198.193:42323)
I20260807 19:49:17.722680 21400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.193@KRBTEST.COM'} at 127.0.198.193:43597
WARNING: no policy specified for kudu/127.0.198.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
Entry for principal kudu/127.0.198.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab.
I20260807 19:49:17.788122 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:18.185075 21601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:18.185438 21601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:18.185689 21601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:18.194134 21601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:18.194306 21601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:18.194437 21601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:18.205302 21601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.194.keytab
--principal=kudu/127.0.198.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:18.206898 21601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:18.208652 21601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:18.225849 21606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:18.225819 21607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:18.226116 21609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:18.226766 21601 server_base.cc:1034] running on GCE node
I20260807 19:49:18.227545 21601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:18.228809 21601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:18.230130 21601 hybrid_clock.cc:648] HybridClock initialized: now 1786132158230071 us; error 82 us; skew 500 ppm
I20260807 19:49:18.230728 21601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:18 dist-test-slave-btw8 krb5kdc[21350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:18.235004 21601 init.cc:377] Logged in from keytab as kudu/127.0.198.194@KRBTEST.COM (short username kudu)
I20260807 19:49:18.236999 21601 webserver.cc:460] Webserver started at http://127.0.198.194:38139/ using document root <none> and password file <none>
I20260807 19:49:18.237828 21601 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:18.238000 21601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:18.238361 21601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:18.241796 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "48e7cc7ee26345a2858a4973d0d36cd9"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "bb5a464c848ebef16b9796979849059b"
server_key_iv: "0a7b748694590303860f89518165412a"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.242722 21601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "48e7cc7ee26345a2858a4973d0d36cd9"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "bb5a464c848ebef16b9796979849059b"
server_key_iv: "0a7b748694590303860f89518165412a"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.249583 21601 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:49:18.254400 21616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:18.256372 21601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:49:18.256632 21601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "48e7cc7ee26345a2858a4973d0d36cd9"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "bb5a464c848ebef16b9796979849059b"
server_key_iv: "0a7b748694590303860f89518165412a"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.256934 21601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:18.274554 21601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:18.281028 21601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:18.281350 21601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:18.282617 21601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:18.284224 21601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:18.284308 21601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:18.284404 21601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:18.284457 21601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:18.326313 21601 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:41561
I20260807 19:49:18.326354 21733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:41561 every 8 connection(s)
I20260807 19:49:18.327932 21601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:18.329084 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21601
I20260807 19:49:18.329279 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:18.329926 795 external_mini_cluster.cc:1428] Setting key 91706c66aea494db41bdbcbdb2632fb1
Aug 07 19:49:18 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.194@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:18.355010 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:18.331264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:33835 (local address 127.0.198.254:37471)
0807 19:49:18.331897 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:18.331905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:18.333244 (+ 1339us) server_negotiation.cc:407] Connection header received
0807 19:49:18.335328 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:18.335334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:18.335612 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:18.335794 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:18.337781 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.338659 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.340803 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.341352 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.341539 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:18.346563 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:18.346569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:18.346572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:18.346615 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:18.350582 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:18.351906 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:18.351911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:18.351914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:18.352120 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:18.352784 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:18.352789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:18.352791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:18.353161 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:18.353302 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:18.354449 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0807 19:49:18.354721 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":328,"threads_started":1}
I20260807 19:49:18.356786 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:18.332081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37471 (local address 127.0.198.194:33835)
0807 19:49:18.333067 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:18.333108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:18.334999 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:18.336110 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:18.336120 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:18.336822 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:18.337553 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:18.337575 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.338893 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.338904 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:18.340559 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:18.340576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.341750 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.341761 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:18.342045 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:18.343039 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:18.343066 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:18.346210 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:18.350826 (+ 4616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:18.350841 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:18.350865 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:18.351690 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:18.352268 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:18.352275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:18.352281 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:18.352585 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:18.353319 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:18.353330 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:18.353907 (+ 577us) client_negotiation.cc:769] Sending connection context
0807 19:49:18.354262 (+ 355us) client_negotiation.cc:240] Negotiation successful
0807 19:49:18.354942 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":468,"threads_started":1}
I20260807 19:49:18.358949 21734 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37471
I20260807 19:49:18.359434 21734 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:18.360476 21734 heartbeater.cc:511] Master 127.0.198.254:37471 requested a full tablet report, sending...
I20260807 19:49:18.362613 21400 ts_manager.cc:194] Registered new tserver with Master: 48e7cc7ee26345a2858a4973d0d36cd9 (127.0.198.194:41561)
I20260807 19:49:18.363819 21400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.194@KRBTEST.COM'} at 127.0.198.194:33835
WARNING: no policy specified for kudu/127.0.198.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
Entry for principal kudu/127.0.198.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab.
I20260807 19:49:18.420945 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:49:18.727607 21595 heartbeater.cc:503] Master 127.0.198.254:37471 was elected leader, sending a full tablet report...
W20260807 19:49:18.810256 21740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:18.810580 21740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:18.810782 21740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:18.819252 21740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:18.819466 21740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:18.819725 21740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:18.830719 21740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/krb5kdc/kudu_127.0.198.195.keytab
--principal=kudu/127.0.198.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:18.832356 21740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:18.833858 21740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:18.849421 21745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:18.849421 21746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:18.849470 21748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:18.850731 21740 server_base.cc:1034] running on GCE node
I20260807 19:49:18.851831 21740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:18.853181 21740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:18.854470 21740 hybrid_clock.cc:648] HybridClock initialized: now 1786132158854417 us; error 56 us; skew 500 ppm
I20260807 19:49:18.854892 21740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:18 dist-test-slave-btw8 krb5kdc[21350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:18.863598 21740 init.cc:377] Logged in from keytab as kudu/127.0.198.195@KRBTEST.COM (short username kudu)
I20260807 19:49:18.865583 21740 webserver.cc:460] Webserver started at http://127.0.198.195:38791/ using document root <none> and password file <none>
I20260807 19:49:18.866447 21740 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:18.866612 21740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:18.867007 21740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:18.870445 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "d5d80936a2f34a7d9f23859428cd7a86"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "236d952cde44fcfc681a8f9770180e04"
server_key_iv: "929db491ea9e8ccc4e7b6a0e355c7723"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.871498 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "d5d80936a2f34a7d9f23859428cd7a86"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "236d952cde44fcfc681a8f9770180e04"
server_key_iv: "929db491ea9e8ccc4e7b6a0e355c7723"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.877552 21740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:18.882370 21755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:18.884564 21740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:49:18.884776 21740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "d5d80936a2f34a7d9f23859428cd7a86"
format_stamp: "Formatted at 2026-08-07 19:49:18 on dist-test-slave-btw8"
server_key: "236d952cde44fcfc681a8f9770180e04"
server_key_iv: "929db491ea9e8ccc4e7b6a0e355c7723"
server_key_version: "encryptionkey@0"
I20260807 19:49:18.885071 21740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:18.901861 21740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:18.907789 21740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:18.908216 21740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:18.909463 21740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:18.911134 21740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:18.911223 21740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:18.911415 21740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:18.911495 21740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 19:49:18.950773 21740 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:38357
I20260807 19:49:18.950821 21877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:38357 every 8 connection(s)
I20260807 19:49:18.952389 21740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:18.953915 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21740
I20260807 19:49:18.954120 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:18.954747 795 external_mini_cluster.cc:1428] Setting key 0947bf06f46ed6d64230a5bd5a32242e
Aug 07 19:49:18 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.195@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:18.982436 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:18.955761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:59319 (local address 127.0.198.254:37471)
0807 19:49:18.959215 (+ 3454us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:18.959222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:18.959841 (+ 619us) server_negotiation.cc:407] Connection header received
0807 19:49:18.962829 (+ 2988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:18.962834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:18.963051 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:18.963172 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:18.965716 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.966625 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.968712 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.969338 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.969470 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:18.975105 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:18.975111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:18.975115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:18.975158 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:18.978305 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:18.979331 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:18.979336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:18.979339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:18.979685 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:18.980261 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:18.980265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:18.980268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:18.980718 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:18.980880 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:18.981893 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0807 19:49:18.982122 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3289,"thread_start_us":3082,"threads_started":1}
I20260807 19:49:18.983894 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:18.956520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:37471 (local address 127.0.198.195:59319)
0807 19:49:18.959637 (+ 3117us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:18.959705 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:18.962397 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:18.963551 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:18.963565 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:18.964464 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:18.965412 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:18.965443 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.966844 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.966850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:18.968446 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:18.968464 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:18.969797 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:18.969813 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:18.970406 (+ 593us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:18.971501 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:49:18.971529 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:18.974538 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:18.978529 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:18.978538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:18.978556 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:18.979141 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:18.979862 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:49:18.979866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:49:18.979869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:49:18.980107 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:49:18.980881 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:18.980888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:18.981372 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 19:49:18.981657 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 19:49:18.982308 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2767,"thread_start_us":2489,"threads_started":1}
I20260807 19:49:18.985806 21878 heartbeater.cc:347] Connected to a master server at 127.0.198.254:37471
I20260807 19:49:18.986232 21878 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:18.987095 21878 heartbeater.cc:511] Master 127.0.198.254:37471 requested a full tablet report, sending...
I20260807 19:49:18.989008 21398 ts_manager.cc:194] Registered new tserver with Master: d5d80936a2f34a7d9f23859428cd7a86 (127.0.198.195:38357)
I20260807 19:49:18.990108 21398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.198.195@KRBTEST.COM'} at 127.0.198.195:59319
I20260807 19:49:18.998451 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:19.018306 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.003046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.0.198.254:37471)
0807 19:49:19.003267 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.003273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.003866 (+ 593us) server_negotiation.cc:407] Connection header received
0807 19:49:19.004115 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.004121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.004215 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.004328 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:19.005864 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.006651 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.008139 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.008643 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.008795 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.011551 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:19.011563 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:19.011568 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:19.011610 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:19.014901 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.015876 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.015883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.015887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.016060 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.016517 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.016523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.016527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.016912 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:19.017238 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.017655 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.017929 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 19:49:19.023654 21398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:19.032393 21398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:19.075695 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.048278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:38357 (local address 127.0.0.1:55032)
0807 19:49:19.058984 (+ 10706us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:19.059068 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:19.065097 (+ 6029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:19.067205 (+ 2108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:19.067212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:19.067251 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:19.067945 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.067961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.070019 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.070024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.072431 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.072447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.074353 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.074361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.074628 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.074701 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 19:49:19.074848 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:49:19.075091 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10250,"mutex_wait_us":67,"thread_start_us":16455,"threads_started":1}
I20260807 19:49:19.077222 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.048742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:42323 (local address 127.0.0.1:48364)
0807 19:49:19.063306 (+ 14564us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:19.063338 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:19.063637 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:19.067143 (+ 3506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:19.067154 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:19.067191 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:19.067935 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.067962 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.070202 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.070211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.073098 (+ 2887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.073116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.076079 (+ 2963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.076087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.076290 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.076328 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:49:19.076428 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:49:19.076605 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14361,"mutex_wait_us":3,"thread_start_us":10632,"threads_started":1}
I20260807 19:49:19.077548 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.048532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.0.198.195:38357)
0807 19:49:19.064619 (+ 16087us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.064633 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.064686 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:49:19.066151 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.066164 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.066513 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.066779 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:19.068188 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.069824 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.072646 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.074107 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.074262 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.076581 (+ 2319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.076692 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.076966 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15812,"thread_start_us":15603,"threads_started":1}
I20260807 19:49:19.077663 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.049598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.0.198.193:42323)
0807 19:49:19.065958 (+ 16360us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.065967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.066002 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:49:19.066122 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.066132 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.066491 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.066768 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:19.068212 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.069999 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.073385 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.075431 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.075616 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.075902 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.076997 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.077253 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16137,"thread_start_us":15937,"threads_started":1}
I20260807 19:49:19.077847 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.064796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:41561 (local address 127.0.0.1:55776)
0807 19:49:19.065632 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:19.065660 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:19.065855 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:19.068636 (+ 2781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:19.068643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:19.068676 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:19.069068 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.069083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.071445 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.071451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.073768 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.073781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.076572 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.076580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.076788 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.076826 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:49:19.076944 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:49:19.077130 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":1,"thread_start_us":1073,"threads_started":1}
I20260807 19:49:19.078373 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.061053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.0.198.194:41561)
0807 19:49:19.067853 (+ 6800us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.067864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.067899 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:49:19.068016 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.068027 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.068364 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.068625 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:19.069299 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.071164 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.073992 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.075990 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.076161 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.076385 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.077501 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.077997 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6591,"thread_start_us":6423,"threads_started":1}
I20260807 19:49:19.082231 21529 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 (DEFAULT_TABLE table=test-table [id=5204757a2f2949c087a3675845e5bdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:19.082237 21807 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 (DEFAULT_TABLE table=test-table [id=5204757a2f2949c087a3675845e5bdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:19.082657 21665 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 (DEFAULT_TABLE table=test-table [id=5204757a2f2949c087a3675845e5bdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:19.084003 21529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6db92d54a04b8fb7fff80b76b19331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:19.084072 21807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6db92d54a04b8fb7fff80b76b19331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:19.084089 21665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6db92d54a04b8fb7fff80b76b19331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:19.099577 21897 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Bootstrap starting.
I20260807 19:49:19.101883 21898 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Bootstrap starting.
I20260807 19:49:19.102969 21899 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Bootstrap starting.
I20260807 19:49:19.104913 21897 tablet_bootstrap.cc:654] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:19.106035 21897 log.cc:824] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:19.106607 21899 tablet_bootstrap.cc:654] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:19.106616 21898 tablet_bootstrap.cc:654] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:19.107910 21898 log.cc:824] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:19.107910 21899 log.cc:824] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:19.109561 21897 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: No bootstrap required, opened a new log
I20260807 19:49:19.109941 21897 ts_tablet_manager.cc:1397] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 19:49:19.110723 21898 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: No bootstrap required, opened a new log
I20260807 19:49:19.111004 21898 ts_tablet_manager.cc:1397] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:49:19.111552 21899 tablet_bootstrap.cc:492] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: No bootstrap required, opened a new log
I20260807 19:49:19.113372 21899 ts_tablet_manager.cc:1397] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260807 19:49:19.118314 21898 raft_consensus.cc:348] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.118940 21898 raft_consensus.cc:374] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:19.119045 21898 raft_consensus.cc:729] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e7cc7ee26345a2858a4973d0d36cd9, State: Initialized, Role: FOLLOWER
I20260807 19:49:19.119767 21898 consensus_queue.cc:260] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.121183 21734 heartbeater.cc:503] Master 127.0.198.254:37471 was elected leader, sending a full tablet report...
I20260807 19:49:19.121773 21898 ts_tablet_manager.cc:1428] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:49:19.122037 21903 raft_consensus.cc:482] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:19.122320 21903 raft_consensus.cc:504] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.122399 21897 raft_consensus.cc:348] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.123059 21897 raft_consensus.cc:374] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:19.123289 21897 raft_consensus.cc:729] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b161dd3834224094bc7242f94299706c, State: Initialized, Role: FOLLOWER
I20260807 19:49:19.124193 21897 consensus_queue.cc:260] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.124887 21899 raft_consensus.cc:348] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.125423 21899 raft_consensus.cc:374] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:19.125558 21899 raft_consensus.cc:729] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d80936a2f34a7d9f23859428cd7a86, State: Initialized, Role: FOLLOWER
I20260807 19:49:19.126394 21899 consensus_queue.cc:260] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.132476 21878 heartbeater.cc:503] Master 127.0.198.254:37471 was elected leader, sending a full tablet report...
I20260807 19:49:19.133615 21903 leader_election.cc:288] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d80936a2f34a7d9f23859428cd7a86 (127.0.198.195:38357), b161dd3834224094bc7242f94299706c (127.0.198.193:42323)
I20260807 19:49:19.137603 21899 ts_tablet_manager.cc:1428] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Time spent starting tablet: real 0.024s user 0.007s sys 0.010s
I20260807 19:49:19.143169 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.125432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:47317 (local address 127.0.198.195:38357)
0807 19:49:19.125687 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.125695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.132934 (+ 7239us) server_negotiation.cc:407] Connection header received
0807 19:49:19.133257 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.133263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.133343 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.133408 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:19.134306 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.135531 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.140783 (+ 5252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.142061 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.142169 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.142224 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.142731 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.142869 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:49:19.143262 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.124757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:38357 (local address 127.0.198.194:47317)
0807 19:49:19.132792 (+ 8035us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:19.132829 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:19.133074 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:19.133651 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:19.133656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:19.133683 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:19.134131 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.134146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.138024 (+ 3878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.138030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.140550 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.140570 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.142413 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.142418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.142545 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.142569 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:49:19.142702 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 19:49:19.142869 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7813,"thread_start_us":14399,"threads_started":1}
I20260807 19:49:19.144543 21828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6db92d54a04b8fb7fff80b76b19331" candidate_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d80936a2f34a7d9f23859428cd7a86" is_pre_election: true
I20260807 19:49:19.145212 21897 ts_tablet_manager.cc:1428] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Time spent starting tablet: real 0.035s user 0.014s sys 0.021s
I20260807 19:49:19.145159 21828 raft_consensus.cc:2454] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e7cc7ee26345a2858a4973d0d36cd9 in term 0.
I20260807 19:49:19.146104 21620 leader_election.cc:302] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e7cc7ee26345a2858a4973d0d36cd9, d5d80936a2f34a7d9f23859428cd7a86; no voters:
I20260807 19:49:19.146664 21903 raft_consensus.cc:2781] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:19.146831 21903 raft_consensus.cc:482] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:19.146937 21903 raft_consensus.cc:3037] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:19.149070 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.132472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:57829 (local address 127.0.198.193:42323)
0807 19:49:19.132869 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.132879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.139110 (+ 6231us) server_negotiation.cc:407] Connection header received
0807 19:49:19.139812 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.139822 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.139935 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.140016 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:19.141170 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.142732 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.146593 (+ 3861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.147951 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.148119 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.148191 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.148673 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.148821 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":78}
I20260807 19:49:19.149362 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.131895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:42323 (local address 127.0.198.194:57829)
0807 19:49:19.138923 (+ 7028us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:19.138966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:19.139554 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:19.140288 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:19.140297 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:19.140333 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:19.140933 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.140948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.143018 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.143027 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.146399 (+ 3372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:19.146416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.148213 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.148225 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:19.148442 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.148480 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:49:19.148642 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 19:49:19.148799 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6794,"thread_start_us":970,"threads_started":1}
I20260807 19:49:19.150364 21903 raft_consensus.cc:504] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.150303 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6db92d54a04b8fb7fff80b76b19331" candidate_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b161dd3834224094bc7242f94299706c" is_pre_election: true
I20260807 19:49:19.150833 21549 raft_consensus.cc:2454] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48e7cc7ee26345a2858a4973d0d36cd9 in term 0.
I20260807 19:49:19.151165 21903 leader_election.cc:288] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [CANDIDATE]: Term 1 election: Requested vote from peers d5d80936a2f34a7d9f23859428cd7a86 (127.0.198.195:38357), b161dd3834224094bc7242f94299706c (127.0.198.193:42323)
I20260807 19:49:19.151541 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6db92d54a04b8fb7fff80b76b19331" candidate_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b161dd3834224094bc7242f94299706c"
I20260807 19:49:19.151540 21828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6db92d54a04b8fb7fff80b76b19331" candidate_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d80936a2f34a7d9f23859428cd7a86"
I20260807 19:49:19.151746 21549 raft_consensus.cc:3037] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:19.151778 21828 raft_consensus.cc:3037] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:19.154181 21549 raft_consensus.cc:2454] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e7cc7ee26345a2858a4973d0d36cd9 in term 1.
I20260807 19:49:19.154188 21828 raft_consensus.cc:2454] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48e7cc7ee26345a2858a4973d0d36cd9 in term 1.
I20260807 19:49:19.154682 21619 leader_election.cc:302] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48e7cc7ee26345a2858a4973d0d36cd9, b161dd3834224094bc7242f94299706c; no voters:
I20260807 19:49:19.155069 21903 raft_consensus.cc:2781] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:19.155710 21903 raft_consensus.cc:686] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 48e7cc7ee26345a2858a4973d0d36cd9, State: Running, Role: LEADER
I20260807 19:49:19.156227 21903 consensus_queue.cc:237] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } }
I20260807 19:49:19.162530 21398 catalog_manager.cc:5495] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48e7cc7ee26345a2858a4973d0d36cd9 (127.0.198.194). New cstate: current_term: 1 leader_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48e7cc7ee26345a2858a4973d0d36cd9" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 41561 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:19.192185 21596 tablet.cc:2369] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:19.192909 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.185897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.0.198.194:41561)
0807 19:49:19.186185 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.186198 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.186220 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:49:19.186439 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.186443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.186546 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.186690 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:19.187154 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.188049 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.189860 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.190455 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.190574 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.191251 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:19.191931 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:19.192049 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.192398 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.192514 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:49:19.202178 21828 raft_consensus.cc:1261] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 1 FOLLOWER]: Refusing update from remote peer 48e7cc7ee26345a2858a4973d0d36cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:19.202181 21549 raft_consensus.cc:1261] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 1 FOLLOWER]: Refusing update from remote peer 48e7cc7ee26345a2858a4973d0d36cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:19.203395 21903 consensus_queue.cc:1035] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b161dd3834224094bc7242f94299706c" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:19.203734 21908 consensus_queue.cc:1035] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d80936a2f34a7d9f23859428cd7a86" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 38357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:49:19.204308 21879 tablet.cc:2369] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:19.236805 21400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:19.237160 21400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:19.241667 21400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f43301e5bf544e9a9126843c630ad86a: Sending DeleteTablet for 3 replicas of tablet 3e6db92d54a04b8fb7fff80b76b19331
I20260807 19:49:19.243553 21529 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:48364
I20260807 19:49:19.243673 21807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:55032
I20260807 19:49:19.244180 21665 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6db92d54a04b8fb7fff80b76b19331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:19 UTC) from {username='kudu', principal='kudu/127.0.198.254@KRBTEST.COM'} at 127.0.0.1:55776
I20260807 19:49:19.249204 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21462
I20260807 19:49:19.260046 21926 tablet_replica.cc:330] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: stopping tablet replica
I20260807 19:49:19.260469 21923 tablet_replica.cc:330] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: stopping tablet replica
I20260807 19:49:19.260633 21926 raft_consensus.cc:2229] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:19.260900 21923 raft_consensus.cc:2229] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:19.261392 21926 raft_consensus.cc:2258] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:19.261495 21923 raft_consensus.cc:2258] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:19.262336 21924 tablet_replica.cc:330] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: stopping tablet replica
I20260807 19:49:19.262814 21924 raft_consensus.cc:2229] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:19.265281 21926 ts_tablet_manager.cc:1905] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:19.265374 21923 ts_tablet_manager.cc:1905] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:19.266281 21924 raft_consensus.cc:2258] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 19:49:19 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.193@KRBTEST.COM
I20260807 19:49:19.272871 21924 ts_tablet_manager.cc:1905] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:19.273907 21923 ts_tablet_manager.cc:1918] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:19.274111 21923 log.cc:1196] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/3e6db92d54a04b8fb7fff80b76b19331
I20260807 19:49:19.274771 21923 ts_tablet_manager.cc:1939] T 3e6db92d54a04b8fb7fff80b76b19331 P d5d80936a2f34a7d9f23859428cd7a86: Deleting consensus metadata
I20260807 19:49:19.277185 21386 catalog_manager.cc:4841] TS d5d80936a2f34a7d9f23859428cd7a86 (127.0.198.195:38357): tablet 3e6db92d54a04b8fb7fff80b76b19331 (table test-table [id=5204757a2f2949c087a3675845e5bdf0]) successfully deleted
I20260807 19:49:19.281286 21926 ts_tablet_manager.cc:1918] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:19.281615 21926 log.cc:1196] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/3e6db92d54a04b8fb7fff80b76b19331
I20260807 19:49:19.281596 21924 ts_tablet_manager.cc:1918] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:19.281869 21924 log.cc:1196] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131811070045-795-0/minicluster-data/ts-0/wal/wals/3e6db92d54a04b8fb7fff80b76b19331
I20260807 19:49:19.282286 21926 ts_tablet_manager.cc:1939] T 3e6db92d54a04b8fb7fff80b76b19331 P 48e7cc7ee26345a2858a4973d0d36cd9: Deleting consensus metadata
I20260807 19:49:19.282505 21924 ts_tablet_manager.cc:1939] T 3e6db92d54a04b8fb7fff80b76b19331 P b161dd3834224094bc7242f94299706c: Deleting consensus metadata
I20260807 19:49:19.283969 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.251672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.0.198.193:42323)
0807 19:49:19.251913 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.251921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.252496 (+ 575us) server_negotiation.cc:407] Connection header received
0807 19:49:19.253630 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.253642 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.253792 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.253904 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:19.259649 (+ 5745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.261089 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.263880 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.264577 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.264805 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.273718 (+ 8913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:19.273737 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:19.273757 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:19.273823 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:19.278858 (+ 5035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.280193 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.280205 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.280208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.280420 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.280977 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.280984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.280988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.281597 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:19.282003 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.283130 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.283481 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:49:19.284412 21385 catalog_manager.cc:4841] TS 48e7cc7ee26345a2858a4973d0d36cd9 (127.0.198.194:41561): tablet 3e6db92d54a04b8fb7fff80b76b19331 (table test-table [id=5204757a2f2949c087a3675845e5bdf0]) successfully deleted
I20260807 19:49:19.285177 21385 catalog_manager.cc:4841] TS b161dd3834224094bc7242f94299706c (127.0.198.193:42323): tablet 3e6db92d54a04b8fb7fff80b76b19331 (table test-table [id=5204757a2f2949c087a3675845e5bdf0]) successfully deleted
I20260807 19:49:19.285858 21590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:19.383296 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21462
I20260807 19:49:19.408751 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21601
Aug 07 19:49:19 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.194@KRBTEST.COM
I20260807 19:49:19.430426 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.409755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.0.198.194:41561)
0807 19:49:19.409971 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.409976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.409995 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:49:19.410334 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.410339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.410441 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.410541 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:19.412511 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.413479 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.416015 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.416661 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.416799 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.422852 (+ 6053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:19.422881 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:19.422903 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:19.422980 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:19.426831 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.427899 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.427908 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.427910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.428102 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.428588 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.428593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.428595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.429000 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:19.429378 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.429713 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.429979 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:49:19.431196 21729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:19.551688 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21601
I20260807 19:49:19.571317 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21740
Aug 07 19:49:19 dist-test-slave-btw8 krb5kdc[21350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.195@KRBTEST.COM
I20260807 19:49:19.595803 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:19.572273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.0.198.195:38357)
0807 19:49:19.572442 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:19.572447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:19.572533 (+ 86us) server_negotiation.cc:407] Connection header received
0807 19:49:19.572704 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:19.572708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:19.572857 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:19.572953 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:19.574550 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.575492 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.577008 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:19.577457 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:19.577578 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:19.588263 (+ 10685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:19.588283 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:19.588300 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:19.588349 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:19.592010 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.593159 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.593168 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.593170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.593401 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:19.593994 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:19.593999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:19.594001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:19.594451 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:19.594768 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:19.595156 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:49:19.595477 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:49:19.596498 21873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:19.716712 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21740
I20260807 19:49:19.737025 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21365
I20260807 19:49:19.738301 21430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:19.854671 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21365
2026-08-07T19:49:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5071 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T19:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:19Z Disabled control of system clock
I20260807 19:49:19.965651 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39683
--webserver_interface=127.0.198.254
--webserver_port=0
--builtin_ntp_servers=127.0.198.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:20.353528 21935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:20.353960 21935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:20.354173 21935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:20.362319 21935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:20.362542 21935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:20.362646 21935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:20.362761 21935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:20.362854 21935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:20.374281 21935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:39683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:20.375973 21935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:20.378042 21935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:20.393292 21941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:20.393393 21943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:20.393275 21940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:20.394843 21935 server_base.cc:1034] running on GCE node
I20260807 19:49:20.395888 21935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:20.397707 21935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:20.398980 21935 hybrid_clock.cc:648] HybridClock initialized: now 1786132160398891 us; error 82 us; skew 500 ppm
I20260807 19:49:20.399648 21935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:20.402477 21935 webserver.cc:460] Webserver started at http://127.0.198.254:43033/ using document root <none> and password file <none>
I20260807 19:49:20.403311 21935 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:20.403537 21935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:20.403921 21935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:20.407250 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "67d9f189f5594d5785d51ff7a588b48a"
format_stamp: "Formatted at 2026-08-07 19:49:20 on dist-test-slave-btw8"
server_key: "4b2a60237af8a8cba82844f2583bcf8a"
server_key_iv: "9c3587aac5e87e75d619bb749d1add40"
server_key_version: "encryptionkey@0"
I20260807 19:49:20.408270 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "67d9f189f5594d5785d51ff7a588b48a"
format_stamp: "Formatted at 2026-08-07 19:49:20 on dist-test-slave-btw8"
server_key: "4b2a60237af8a8cba82844f2583bcf8a"
server_key_iv: "9c3587aac5e87e75d619bb749d1add40"
server_key_version: "encryptionkey@0"
I20260807 19:49:20.414247 21935 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:20.418860 21949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:20.420379 21935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:49:20.420608 21935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "67d9f189f5594d5785d51ff7a588b48a"
format_stamp: "Formatted at 2026-08-07 19:49:20 on dist-test-slave-btw8"
server_key: "4b2a60237af8a8cba82844f2583bcf8a"
server_key_iv: "9c3587aac5e87e75d619bb749d1add40"
server_key_version: "encryptionkey@0"
I20260807 19:49:20.420928 21935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:20.469897 21935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:20.470929 21935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:20.471299 21935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:20.494992 21935 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:39683
I20260807 19:49:20.495042 22004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:39683 every 8 connection(s)
I20260807 19:49:20.496645 21935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:20.497407 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 21935
I20260807 19:49:20.497639 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:20.498296 795 external_mini_cluster.cc:1428] Setting key 61004a0950d282e182026ed87211e5a0
I20260807 19:49:20.501957 22005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:20.514751 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:20.500473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.0.198.254:39683)
0807 19:49:20.501581 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:20.501592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:20.501636 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:49:20.503570 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:20.503594 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:20.504091 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:20.504542 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:20.505412 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:20.506934 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:20.509008 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:20.509800 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:20.510074 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:20.511779 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:20.511789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:20.511811 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:20.512497 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:20.512595 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:20.512623 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:20.512816 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:20.513082 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 19:49:20.513670 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":419,"threads_started":1}
I20260807 19:49:20.516270 22005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: Bootstrap starting.
I20260807 19:49:20.520328 22005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:20.521451 22005 log.cc:824] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:20.525395 22005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: No bootstrap required, opened a new log
I20260807 19:49:20.536036 22005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } }
I20260807 19:49:20.537145 22005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:20.537338 22005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d9f189f5594d5785d51ff7a588b48a, State: Initialized, Role: FOLLOWER
I20260807 19:49:20.538197 22005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } }
I20260807 19:49:20.538523 22005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:20.538714 22005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:20.538973 22005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:20.541528 22005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } }
I20260807 19:49:20.542131 22005 leader_election.cc:302] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d9f189f5594d5785d51ff7a588b48a; no voters:
I20260807 19:49:20.542992 22005 leader_election.cc:288] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:20.543291 22010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:20.545240 22010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [term 1 LEADER]: Becoming Leader. State: Replica: 67d9f189f5594d5785d51ff7a588b48a, State: Running, Role: LEADER
I20260807 19:49:20.545619 22005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:20.545974 22010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } }
I20260807 19:49:20.550806 22011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67d9f189f5594d5785d51ff7a588b48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } } }
I20260807 19:49:20.551544 22011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:20.549964 22012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d9f189f5594d5785d51ff7a588b48a. Latest consensus state: current_term: 1 leader_uuid: "67d9f189f5594d5785d51ff7a588b48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d9f189f5594d5785d51ff7a588b48a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 39683 } } }
I20260807 19:49:20.552057 22012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:20.553620 22019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:20.560197 22019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:20.571756 22019 catalog_manager.cc:1347] Generated new cluster ID: 52d9a8196c374490b7ff07a986bce0aa
I20260807 19:49:20.571904 22019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:20.592180 22019 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:20.594175 22019 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:20.608125 22019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: Generated new TSK 0
I20260807 19:49:20.609318 22019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:20.623775 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.193:0
--local_ip_for_outbound_sockets=127.0.198.193
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--builtin_ntp_servers=127.0.198.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:49:21.032621 22029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:21.032992 22029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:21.033135 22029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:21.041215 22029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:21.041358 22029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:21.041549 22029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.193
I20260807 19:49:21.053500 22029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.198.193
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.193
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:21.055789 22029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:21.058175 22029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:21.075065 22037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:21.074990 22034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:21.075111 22035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:21.076874 22029 server_base.cc:1034] running on GCE node
I20260807 19:49:21.077670 22029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:21.078879 22029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:21.080197 22029 hybrid_clock.cc:648] HybridClock initialized: now 1786132161080114 us; error 77 us; skew 500 ppm
I20260807 19:49:21.080629 22029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:21.083671 22029 webserver.cc:460] Webserver started at http://127.0.198.193:35577/ using document root <none> and password file <none>
I20260807 19:49:21.084580 22029 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:21.084759 22029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:21.085215 22029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:21.089274 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data/instance:
uuid: "3bafb97696ee45cfa1151a37f953a660"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "414a4b3303f4971dcf7adfb97d83a8a9"
server_key_iv: "1a7d30a1d23e2460bcc3454e56274d83"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.090476 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal/instance:
uuid: "3bafb97696ee45cfa1151a37f953a660"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "414a4b3303f4971dcf7adfb97d83a8a9"
server_key_iv: "1a7d30a1d23e2460bcc3454e56274d83"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.097481 22029 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:49:21.102286 22043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.104331 22029 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:49:21.104592 22029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal
uuid: "3bafb97696ee45cfa1151a37f953a660"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "414a4b3303f4971dcf7adfb97d83a8a9"
server_key_iv: "1a7d30a1d23e2460bcc3454e56274d83"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.104900 22029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:21.127450 22029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:21.128485 22029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:21.129050 22029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:21.130535 22029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:21.132876 22029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:21.133062 22029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.133229 22029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:21.133338 22029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.177497 22029 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.193:45875
I20260807 19:49:21.177726 22161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.193:45875 every 8 connection(s)
I20260807 19:49:21.179030 22029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/data/info.pb
I20260807 19:49:21.188570 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22029
I20260807 19:49:21.188793 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-0/wal/instance
I20260807 19:49:21.189313 795 external_mini_cluster.cc:1428] Setting key 6b60611929debd37e550f59357a98283
I20260807 19:49:21.193203 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.194:0
--local_ip_for_outbound_sockets=127.0.198.194
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--builtin_ntp_servers=127.0.198.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:21.197177 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:21.182103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.193:43825 (local address 127.0.198.254:39683)
0807 19:49:21.182709 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:21.182716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:21.184054 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 19:49:21.186370 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:21.186379 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:21.186516 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:21.186721 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:21.188515 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.189470 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.192163 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.192601 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.192772 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:21.195866 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:21.195873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:21.195877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:21.195906 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:21.195954 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:21.195968 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:21.196091 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:21.196391 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 19:49:21.196532 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":288,"threads_started":1}
I20260807 19:49:21.198549 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:21.182752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39683 (local address 127.0.198.193:43825)
0807 19:49:21.183877 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:21.183919 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:21.185863 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:21.187132 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:21.187145 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:21.187292 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:21.188223 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:21.188249 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.189708 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.189713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:21.191945 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:21.191968 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.192902 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.192911 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:21.193230 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:21.195240 (+ 2010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:21.195244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:21.195309 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:21.195544 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:21.196091 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:21.196110 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:21.196121 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:49:21.196374 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 19:49:21.197064 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":306,"threads_started":1}
I20260807 19:49:21.200825 22162 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39683
I20260807 19:49:21.201252 22162 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:21.202252 22162 heartbeater.cc:511] Master 127.0.198.254:39683 requested a full tablet report, sending...
I20260807 19:49:21.205044 21967 ts_manager.cc:194] Registered new tserver with Master: 3bafb97696ee45cfa1151a37f953a660 (127.0.198.193:45875)
I20260807 19:49:21.207487 21967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.193:43825
W20260807 19:49:21.598920 22166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:21.599331 22166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:21.599519 22166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:21.607268 22166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:21.607491 22166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:21.607686 22166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.194
I20260807 19:49:21.619004 22166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.198.194
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.194
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:21.620512 22166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:21.622519 22166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:21.639628 22172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:21.639547 22173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:21.639539 22175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:21.640426 22166 server_base.cc:1034] running on GCE node
I20260807 19:49:21.641224 22166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:21.642481 22166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:21.643748 22166 hybrid_clock.cc:648] HybridClock initialized: now 1786132161643674 us; error 65 us; skew 500 ppm
I20260807 19:49:21.644150 22166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:21.646991 22166 webserver.cc:460] Webserver started at http://127.0.198.194:38825/ using document root <none> and password file <none>
I20260807 19:49:21.647871 22166 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:21.648016 22166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:21.648377 22166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:21.651703 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data/instance:
uuid: "128405e8e4464313aa0043b070045793"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "6963b8d2834bce199e6abd0a8b091962"
server_key_iv: "4037a59ac602f4e330ea2822390401d8"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.652727 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal/instance:
uuid: "128405e8e4464313aa0043b070045793"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "6963b8d2834bce199e6abd0a8b091962"
server_key_iv: "4037a59ac602f4e330ea2822390401d8"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.659627 22166 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:49:21.663946 22182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.665656 22166 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:49:21.665899 22166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal
uuid: "128405e8e4464313aa0043b070045793"
format_stamp: "Formatted at 2026-08-07 19:49:21 on dist-test-slave-btw8"
server_key: "6963b8d2834bce199e6abd0a8b091962"
server_key_iv: "4037a59ac602f4e330ea2822390401d8"
server_key_version: "encryptionkey@0"
I20260807 19:49:21.666177 22166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:21.684659 22166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:21.685745 22166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:21.686115 22166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:21.687279 22166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:21.689211 22166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:21.689328 22166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.689508 22166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:21.689628 22166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:21.728013 22166 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.194:44495
I20260807 19:49:21.728117 22301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.194:44495 every 8 connection(s)
I20260807 19:49:21.729440 22166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/data/info.pb
I20260807 19:49:21.737142 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22166
I20260807 19:49:21.737358 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal/instance
I20260807 19:49:21.737826 795 external_mini_cluster.cc:1428] Setting key 434992f8a961e433b4409720a1233348
I20260807 19:49:21.741391 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.198.195:0
--local_ip_for_outbound_sockets=127.0.198.195
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--builtin_ntp_servers=127.0.198.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:49:21.745381 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:21.732506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.194:35329 (local address 127.0.198.254:39683)
0807 19:49:21.733306 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:21.733311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:21.734497 (+ 1186us) server_negotiation.cc:407] Connection header received
0807 19:49:21.736590 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:21.736596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:21.736711 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:21.736842 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:21.738308 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.739295 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.741654 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.742406 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.742570 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:21.744268 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:21.744277 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:21.744284 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:21.744315 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:21.744372 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:21.744391 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:21.744493 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:21.744773 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:49:21.744904 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":284,"threads_started":1}
I20260807 19:49:21.746476 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:21.733175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39683 (local address 127.0.198.194:35329)
0807 19:49:21.734295 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:21.734342 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:21.736261 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:21.737100 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:21.737109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:21.737220 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:21.738105 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:21.738127 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.739541 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.739547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:21.741408 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:21.741425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:21.742685 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:21.742691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:21.742948 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:21.743845 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:21.743848 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:21.743894 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:21.743987 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:21.744520 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:21.744531 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:21.744538 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:49:21.744725 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 19:49:21.745125 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":385,"threads_started":1}
I20260807 19:49:21.748479 22302 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39683
I20260807 19:49:21.748886 22302 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:21.750195 22302 heartbeater.cc:511] Master 127.0.198.254:39683 requested a full tablet report, sending...
I20260807 19:49:21.752661 21967 ts_manager.cc:194] Registered new tserver with Master: 128405e8e4464313aa0043b070045793 (127.0.198.194:44495)
I20260807 19:49:21.753681 21967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.194:35329
W20260807 19:49:22.131132 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:22.131577 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:22.131680 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:22.139513 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:22.139639 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:22.139752 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.198.195
I20260807 19:49:22.149773 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.198.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.198.195
--webserver_port=0
--tserver_master_addrs=127.0.198.254:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.198.195
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:22.151135 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:22.153328 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:22.169464 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:22.169452 22314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:22.169389 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:22.170238 22306 server_base.cc:1034] running on GCE node
I20260807 19:49:22.170943 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:22.172263 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:22.173534 22306 hybrid_clock.cc:648] HybridClock initialized: now 1786132162173497 us; error 70 us; skew 500 ppm
I20260807 19:49:22.173919 22306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:49:22.176707 22306 webserver.cc:460] Webserver started at http://127.0.198.195:40817/ using document root <none> and password file <none>
I20260807 19:49:22.177567 22306 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:22.177719 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:22.178079 22306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:22.181218 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data/instance:
uuid: "791201bfd892449e9f359f19eb554f30"
format_stamp: "Formatted at 2026-08-07 19:49:22 on dist-test-slave-btw8"
server_key: "0a4d6e4d8b26c5ae7ec876f4a37d3f5a"
server_key_iv: "685ff6eec15efea914ea835434ecc1dd"
server_key_version: "encryptionkey@0"
I20260807 19:49:22.182245 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal/instance:
uuid: "791201bfd892449e9f359f19eb554f30"
format_stamp: "Formatted at 2026-08-07 19:49:22 on dist-test-slave-btw8"
server_key: "0a4d6e4d8b26c5ae7ec876f4a37d3f5a"
server_key_iv: "685ff6eec15efea914ea835434ecc1dd"
server_key_version: "encryptionkey@0"
I20260807 19:49:22.188189 22306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:49:22.192303 22321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:22.193912 22306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:49:22.194133 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal
uuid: "791201bfd892449e9f359f19eb554f30"
format_stamp: "Formatted at 2026-08-07 19:49:22 on dist-test-slave-btw8"
server_key: "0a4d6e4d8b26c5ae7ec876f4a37d3f5a"
server_key_iv: "685ff6eec15efea914ea835434ecc1dd"
server_key_version: "encryptionkey@0"
I20260807 19:49:22.194428 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:22.212074 22162 heartbeater.cc:503] Master 127.0.198.254:39683 was elected leader, sending a full tablet report...
I20260807 19:49:22.231518 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:22.232517 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:22.232920 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:22.234159 22306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:49:22.235991 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:49:22.236102 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:22.236235 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:49:22.236316 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:22.276242 22306 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.195:45373
I20260807 19:49:22.276307 22441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.195:45373 every 8 connection(s)
I20260807 19:49:22.277725 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/data/info.pb
I20260807 19:49:22.285003 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22306
I20260807 19:49:22.285229 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal/instance
I20260807 19:49:22.285720 795 external_mini_cluster.cc:1428] Setting key 20674467a10cef8454e25cde89571570
I20260807 19:49:22.292495 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.280740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:37723 (local address 127.0.198.254:39683)
0807 19:49:22.281215 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.281220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.282612 (+ 1392us) server_negotiation.cc:407] Connection header received
0807 19:49:22.284691 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.284697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.284802 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.284933 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:22.286324 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.287393 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.289419 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.289908 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.290020 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.291634 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:22.291642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:22.291645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:22.291668 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:22.291713 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:22.291726 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:22.291808 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.292073 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.292207 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260807 19:49:22.293557 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.281399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:39683 (local address 127.0.198.195:37723)
0807 19:49:22.282428 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.282476 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.284352 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.285234 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.285248 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.285384 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:49:22.286093 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.286113 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.287591 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.287596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.289217 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.289246 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.290141 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.290146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.290372 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.291152 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:49:22.291155 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:49:22.291202 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:49:22.291337 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:49:22.291830 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:49:22.291838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:49:22.291845 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.292008 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.292358 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":361,"threads_started":1}
I20260807 19:49:22.295563 22442 heartbeater.cc:347] Connected to a master server at 127.0.198.254:39683
I20260807 19:49:22.296036 22442 heartbeater.cc:464] Registering TS with master...
I20260807 19:49:22.296949 22442 heartbeater.cc:511] Master 127.0.198.254:39683 requested a full tablet report, sending...
I20260807 19:49:22.299291 21967 ts_manager.cc:194] Registered new tserver with Master: 791201bfd892449e9f359f19eb554f30 (127.0.198.195:45373)
I20260807 19:49:22.300570 21967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.198.195:37723
I20260807 19:49:22.302322 795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:49:22.313696 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.306863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.0.198.254:39683)
0807 19:49:22.307106 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.307117 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.307612 (+ 495us) server_negotiation.cc:407] Connection header received
0807 19:49:22.307730 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.307735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.307815 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.307903 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:22.308405 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.309175 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.310737 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.311163 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.311291 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.312549 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:22.312554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:22.312557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:22.312579 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:22.312621 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:22.312639 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:22.312721 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.312946 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.313153 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:49:22.318661 21967 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:49:22.326290 21967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:49:22.355338 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.340722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50486)
0807 19:49:22.342004 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.342063 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.342943 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.344197 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.344209 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.344251 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:22.344708 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.344735 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.347264 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.347274 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.350590 (+ 3316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.350624 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.354019 (+ 3395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.354031 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.354345 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.354412 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.354572 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.354945 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":573,"threads_started":1}
I20260807 19:49:22.356966 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.341003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.0.198.193:45875)
0807 19:49:22.342680 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.342691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.342741 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:49:22.343251 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.343262 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.343603 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.343831 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:22.344984 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.347012 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.350988 (+ 3976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.353151 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.353354 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.353795 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.356100 (+ 2305us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.356424 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1465,"thread_start_us":1311,"threads_started":1}
I20260807 19:49:22.362833 22094 tablet_service.cc:1467] Processing CreateTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 (DEFAULT_TABLE table=test-table [id=9cd1f9e6fbda4c5583f118998ec78f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:22.364616 22094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6066aeaa8a71475d83640ff07bed75d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:22.365444 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.343040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.0.198.194:44495)
0807 19:49:22.355043 (+ 12003us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.355056 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.355877 (+ 821us) server_negotiation.cc:407] Connection header received
0807 19:49:22.356041 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.356055 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.356444 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.356727 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:22.357844 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.360014 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.361605 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.363141 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.363290 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.363570 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.364890 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.365099 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10683,"thread_start_us":11602,"threads_started":1}
I20260807 19:49:22.365742 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.342470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44495 (local address 127.0.0.1:47724)
0807 19:49:22.355700 (+ 13230us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.355721 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.355844 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.357314 (+ 1470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.357319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.357352 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:22.357584 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.357604 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.360183 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.360188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.361368 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.361423 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.363801 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.363808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.364035 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.364083 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.364190 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.364349 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13056,"mutex_wait_us":4,"thread_start_us":15241,"threads_started":1}
I20260807 19:49:22.366392 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.342459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.195:45373 (local address 127.0.0.1:57698)
0807 19:49:22.354512 (+ 12053us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.355114 (+ 602us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.355300 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.358718 (+ 3418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.358723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.358757 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:22.359225 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.359239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.361483 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.361490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.363501 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.363521 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.365413 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.365420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.365626 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.365680 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.365779 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.365937 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11839,"thread_start_us":11641,"threads_started":1}
I20260807 19:49:22.366468 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.343128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.0.198.195:45373)
0807 19:49:22.357700 (+ 14572us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.357714 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.357751 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:49:22.357855 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.357863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.358168 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.358408 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:22.359498 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.361284 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.363771 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.365185 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.365370 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.365670 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.365863 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.366034 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14320,"thread_start_us":14139,"threads_started":1}
I20260807 19:49:22.370500 22233 tablet_service.cc:1467] Processing CreateTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 (DEFAULT_TABLE table=test-table [id=9cd1f9e6fbda4c5583f118998ec78f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:22.370492 22375 tablet_service.cc:1467] Processing CreateTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 (DEFAULT_TABLE table=test-table [id=9cd1f9e6fbda4c5583f118998ec78f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:49:22.372035 22233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6066aeaa8a71475d83640ff07bed75d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:22.372105 22375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6066aeaa8a71475d83640ff07bed75d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:22.381254 22462 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Bootstrap starting.
I20260807 19:49:22.386801 22462 tablet_bootstrap.cc:654] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:22.387096 22464 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Bootstrap starting.
I20260807 19:49:22.388183 22463 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Bootstrap starting.
I20260807 19:49:22.389883 22462 log.cc:824] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:22.391095 22464 tablet_bootstrap.cc:654] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:22.392340 22463 tablet_bootstrap.cc:654] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:22.392622 22464 log.cc:824] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:22.393787 22463 log.cc:824] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:22.394044 22462 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: No bootstrap required, opened a new log
I20260807 19:49:22.394395 22462 ts_tablet_manager.cc:1397] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 19:49:22.396823 22464 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: No bootstrap required, opened a new log
I20260807 19:49:22.397135 22464 ts_tablet_manager.cc:1397] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:49:22.397656 22463 tablet_bootstrap.cc:492] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: No bootstrap required, opened a new log
I20260807 19:49:22.397977 22463 ts_tablet_manager.cc:1397] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 19:49:22.406046 22462 raft_consensus.cc:348] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.406620 22462 raft_consensus.cc:374] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:22.406733 22462 raft_consensus.cc:729] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bafb97696ee45cfa1151a37f953a660, State: Initialized, Role: FOLLOWER
I20260807 19:49:22.406992 22463 raft_consensus.cc:348] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.407522 22463 raft_consensus.cc:374] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:22.407473 22462 consensus_queue.cc:260] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.407672 22463 raft_consensus.cc:729] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791201bfd892449e9f359f19eb554f30, State: Initialized, Role: FOLLOWER
I20260807 19:49:22.408340 22463 consensus_queue.cc:260] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.408420 22464 raft_consensus.cc:348] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.408869 22464 raft_consensus.cc:374] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:22.408960 22464 raft_consensus.cc:729] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128405e8e4464313aa0043b070045793, State: Initialized, Role: FOLLOWER
I20260807 19:49:22.409694 22462 ts_tablet_manager.cc:1428] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 19:49:22.409643 22464 consensus_queue.cc:260] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.410920 22442 heartbeater.cc:503] Master 127.0.198.254:39683 was elected leader, sending a full tablet report...
I20260807 19:49:22.411494 22302 heartbeater.cc:503] Master 127.0.198.254:39683 was elected leader, sending a full tablet report...
I20260807 19:49:22.412150 22464 ts_tablet_manager.cc:1428] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 19:49:22.413383 22463 ts_tablet_manager.cc:1428] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260807 19:49:22.414992 22469 raft_consensus.cc:482] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:22.415320 22469 raft_consensus.cc:504] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.417697 22469 leader_election.cc:288] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bafb97696ee45cfa1151a37f953a660 (127.0.198.193:45875), 128405e8e4464313aa0043b070045793 (127.0.198.194:44495)
I20260807 19:49:22.424335 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.418766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:33873 (local address 127.0.198.193:45875)
0807 19:49:22.418973 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.418982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.419008 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:49:22.419081 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.419086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.419197 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.419301 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:22.419997 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.420740 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.422473 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.423292 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.423543 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.423621 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.423933 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.424088 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:49:22.424391 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.418017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.198.195:33873)
0807 19:49:22.418212 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.418235 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.418361 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.419572 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.419576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.419601 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:22.419845 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.419856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.420919 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.420925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.422283 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.422299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.423542 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.423548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.423753 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.423779 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.423903 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.424058 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"spinlock_wait_cycles":2304}
I20260807 19:49:22.425490 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.418428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.194:44495 (local address 127.0.198.195:44911)
0807 19:49:22.419425 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.419447 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:49:22.419645 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:49:22.420042 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:49:22.420047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:49:22.420072 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:49:22.420484 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.420494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.421621 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.421627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.423206 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:49:22.423219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.424831 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.424837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:49:22.425022 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.425060 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:49:22.425161 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 19:49:22.425276 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":905,"threads_started":1}
I20260807 19:49:22.425791 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.419065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.198.195:44911 (local address 127.0.198.194:44495)
0807 19:49:22.419279 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.419287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.419532 (+ 245us) server_negotiation.cc:407] Connection header received
0807 19:49:22.419800 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.419806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.419903 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.420021 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:49:22.420679 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.421440 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.423476 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.424358 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.424577 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.424681 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.425247 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.425425 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:49:22.425838 22115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6066aeaa8a71475d83640ff07bed75d9" candidate_uuid: "791201bfd892449e9f359f19eb554f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bafb97696ee45cfa1151a37f953a660" is_pre_election: true
I20260807 19:49:22.426527 22115 raft_consensus.cc:2454] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791201bfd892449e9f359f19eb554f30 in term 0.
I20260807 19:49:22.426792 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6066aeaa8a71475d83640ff07bed75d9" candidate_uuid: "791201bfd892449e9f359f19eb554f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128405e8e4464313aa0043b070045793" is_pre_election: true
I20260807 19:49:22.427309 22255 raft_consensus.cc:2454] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791201bfd892449e9f359f19eb554f30 in term 0.
I20260807 19:49:22.427402 22324 leader_election.cc:302] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bafb97696ee45cfa1151a37f953a660, 791201bfd892449e9f359f19eb554f30; no voters:
I20260807 19:49:22.427949 22469 raft_consensus.cc:2781] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:49:22.428113 22469 raft_consensus.cc:482] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:22.428216 22469 raft_consensus.cc:3037] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:22.430532 22469 raft_consensus.cc:504] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.431236 22469 leader_election.cc:288] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [CANDIDATE]: Term 1 election: Requested vote from peers 3bafb97696ee45cfa1151a37f953a660 (127.0.198.193:45875), 128405e8e4464313aa0043b070045793 (127.0.198.194:44495)
I20260807 19:49:22.432106 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6066aeaa8a71475d83640ff07bed75d9" candidate_uuid: "791201bfd892449e9f359f19eb554f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128405e8e4464313aa0043b070045793"
I20260807 19:49:22.432124 22115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6066aeaa8a71475d83640ff07bed75d9" candidate_uuid: "791201bfd892449e9f359f19eb554f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bafb97696ee45cfa1151a37f953a660"
W20260807 19:49:22.432184 22163 tablet.cc:2369] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:22.432379 22115 raft_consensus.cc:3037] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:22.432371 22255 raft_consensus.cc:3037] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:22.435963 22115 raft_consensus.cc:2454] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791201bfd892449e9f359f19eb554f30 in term 1.
I20260807 19:49:22.435962 22255 raft_consensus.cc:2454] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791201bfd892449e9f359f19eb554f30 in term 1.
I20260807 19:49:22.436532 22324 leader_election.cc:302] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128405e8e4464313aa0043b070045793, 791201bfd892449e9f359f19eb554f30; no voters:
I20260807 19:49:22.436982 22469 raft_consensus.cc:2781] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:22.437556 22469 raft_consensus.cc:686] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 1 LEADER]: Becoming Leader. State: Replica: 791201bfd892449e9f359f19eb554f30, State: Running, Role: LEADER
I20260807 19:49:22.438127 22469 consensus_queue.cc:237] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } }
I20260807 19:49:22.443234 21966 catalog_manager.cc:5495] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 791201bfd892449e9f359f19eb554f30 (127.0.198.195). New cstate: current_term: 1 leader_uuid: "791201bfd892449e9f359f19eb554f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791201bfd892449e9f359f19eb554f30" member_type: VOTER last_known_addr { host: "127.0.198.195" port: 45373 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:22.470150 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.464179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.0.198.195:45373)
0807 19:49:22.464412 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.464421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.464447 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:49:22.464762 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.464768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.464863 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.465046 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:49:22.465445 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.466231 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.467801 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.468264 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.468420 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.468972 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:49:22.469520 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:49:22.469730 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.469785 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.469879 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 19:49:22.480902 22255 raft_consensus.cc:1261] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 1 FOLLOWER]: Refusing update from remote peer 791201bfd892449e9f359f19eb554f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:22.480919 22115 raft_consensus.cc:1261] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660 [term 1 FOLLOWER]: Refusing update from remote peer 791201bfd892449e9f359f19eb554f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 19:49:22.482167 22303 tablet.cc:2369] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:49:22.482264 22473 consensus_queue.cc:1035] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bafb97696ee45cfa1151a37f953a660" member_type: VOTER last_known_addr { host: "127.0.198.193" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:22.482672 22469 consensus_queue.cc:1035] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128405e8e4464313aa0043b070045793" member_type: VOTER last_known_addr { host: "127.0.198.194" port: 44495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:49:22.495018 22476 mvcc.cc:205] Tried to move back new op lower bound from 7315997337507610624 to 7315997337351086080. Current Snapshot: MvccSnapshot[applied={T|T < 7315997337507610624}]
I20260807 19:49:22.529855 21966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:22.530309 21966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:49:22.537191 21966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67d9f189f5594d5785d51ff7a588b48a: Sending DeleteTablet for 3 replicas of tablet 6066aeaa8a71475d83640ff07bed75d9
I20260807 19:49:22.539893 22094 tablet_service.cc:1514] Processing DeleteTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:22 UTC) from {username='slave'} at 127.0.0.1:50486
I20260807 19:49:22.540302 22375 tablet_service.cc:1514] Processing DeleteTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:22 UTC) from {username='slave'} at 127.0.0.1:57698
I20260807 19:49:22.541061 22233 tablet_service.cc:1514] Processing DeleteTablet for tablet 6066aeaa8a71475d83640ff07bed75d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:49:22 UTC) from {username='slave'} at 127.0.0.1:47724
I20260807 19:49:22.548782 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22029
I20260807 19:49:22.563548 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.549967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.0.198.193:45875)
0807 19:49:22.550250 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.550258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.551399 (+ 1141us) server_negotiation.cc:407] Connection header received
0807 19:49:22.551814 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.551824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.551977 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.552082 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:22.553273 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.554874 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.557698 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.558694 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.558853 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.561057 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:22.561075 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:22.561102 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:22.562170 (+ 1068us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:22.562262 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:22.562301 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:22.562418 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.562747 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.562962 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:49:22.565022 22487 tablet_replica.cc:330] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: stopping tablet replica
I20260807 19:49:22.565217 22157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:22.565701 22487 raft_consensus.cc:2229] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:49:22.566607 22487 raft_consensus.cc:2258] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:22.567399 22488 tablet_replica.cc:330] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: stopping tablet replica
I20260807 19:49:22.567883 22488 raft_consensus.cc:2229] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:22.568444 22488 raft_consensus.cc:2258] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:22.572064 22488 ts_tablet_manager.cc:1905] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:22.577270 22487 ts_tablet_manager.cc:1905] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:49:22.594296 22488 ts_tablet_manager.cc:1918] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:22.594547 22488 log.cc:1196] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-1/wal/wals/6066aeaa8a71475d83640ff07bed75d9
I20260807 19:49:22.595180 22488 ts_tablet_manager.cc:1939] T 6066aeaa8a71475d83640ff07bed75d9 P 128405e8e4464313aa0043b070045793: Deleting consensus metadata
I20260807 19:49:22.597862 21952 catalog_manager.cc:4841] TS 128405e8e4464313aa0043b070045793 (127.0.198.194:44495): tablet 6066aeaa8a71475d83640ff07bed75d9 (table test-table [id=9cd1f9e6fbda4c5583f118998ec78f84]) successfully deleted
I20260807 19:49:22.599200 22487 ts_tablet_manager.cc:1918] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:49:22.599588 22487 log.cc:1196] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Deleting WAL directory at /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131811070045-795-0/minicluster-data/ts-2/wal/wals/6066aeaa8a71475d83640ff07bed75d9
I20260807 19:49:22.600229 22487 ts_tablet_manager.cc:1939] T 6066aeaa8a71475d83640ff07bed75d9 P 791201bfd892449e9f359f19eb554f30: Deleting consensus metadata
I20260807 19:49:22.602384 21951 catalog_manager.cc:4841] TS 791201bfd892449e9f359f19eb554f30 (127.0.198.195:45373): tablet 6066aeaa8a71475d83640ff07bed75d9 (table test-table [id=9cd1f9e6fbda4c5583f118998ec78f84]) successfully deleted
I20260807 19:49:22.689509 22486 tablet_replica.cc:330] T 6066aeaa8a71475d83640ff07bed75d9 P 3bafb97696ee45cfa1151a37f953a660: stopping tablet replica
I20260807 19:49:22.689837 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22029
I20260807 19:49:22.712352 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22166
W20260807 19:49:22.712373 21952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.198.193:45875 (error 108)
I20260807 19:49:22.714810 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.714153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50502)
0807 19:49:22.714472 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.714609 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260807 19:49:22.715142 21952 catalog_manager.cc:4584] TS 3bafb97696ee45cfa1151a37f953a660 (127.0.198.193:45875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6066aeaa8a71475d83640ff07bed75d9: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
I20260807 19:49:22.721007 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.713603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.0.198.194:44495)
0807 19:49:22.713813 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.713820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.713842 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:49:22.713913 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.713920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.714028 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.714150 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:22.714669 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.715722 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.717089 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.717543 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.717709 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.719057 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:22.719068 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:22.719084 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:22.719997 (+ 913us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:22.720075 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:22.720102 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:22.720256 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.720443 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.720585 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:49:22.721871 22297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:22.769518 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.768752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50504)
0807 19:49:22.769150 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.769284 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260807 19:49:22.771018 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.770545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50512)
0807 19:49:22.770741 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.770847 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:49:22.820068 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22166
I20260807 19:49:22.832410 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.831680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50526)
0807 19:49:22.831975 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.832133 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:49:22.834025 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.833519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50530)
0807 19:49:22.833740 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.833851 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:49:22.844235 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22306
I20260807 19:49:22.854470 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.845433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.0.198.195:45373)
0807 19:49:22.845675 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:22.845683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:22.845705 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:49:22.845778 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:22.845791 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:22.845903 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:22.846043 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:22.846645 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.848050 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.849661 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:22.850603 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:22.850836 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:22.852209 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:22.852226 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:22.852250 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:22.853380 (+ 1130us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:22.853460 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:22.853497 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:22.853629 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:22.853918 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 19:49:22.854054 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 19:49:22.855852 22437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:22.914655 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.914023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50534)
0807 19:49:22.914311 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.914428 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 19:49:22.916059 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:22.915512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.193:45875 (local address 127.0.0.1:50550)
0807 19:49:22.915789 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 19:49:22.915895 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.193:45875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 19:49:22.991449 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22306
I20260807 19:49:23.014991 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 21935
I20260807 19:49:23.016438 22000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:23.137298 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 21935
2026-08-07T19:49:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3281 ms)
[----------] 6 tests from EncryptionPolicyTest (25143 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:23 dist-test-slave-btw8 krb5kdc[22497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:23 dist-test-slave-btw8 krb5kdc[22497](info): set up 2 sockets
Aug 07 19:49:23 dist-test-slave-btw8 krb5kdc[22497](info): commencing operation
krb5kdc: starting...
W20260807 19:49:24.380744 795 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:24 dist-test-slave-btw8 krb5kdc[22497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:24.784793 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:25.166826 22511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:25.167219 22511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:25.167440 22511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:25.176642 22511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:25.176815 22511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:25.176909 22511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:25.177017 22511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:25.177103 22511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:25.188431 22511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:25.189975 22511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:25.191468 22511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:25.205415 22519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:25.205500 22516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:25.205438 22517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:25.206193 22511 server_base.cc:1034] running on GCE node
I20260807 19:49:25.207295 22511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:25.209100 22511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:25.210358 22511 hybrid_clock.cc:648] HybridClock initialized: now 1786132165210310 us; error 43 us; skew 500 ppm
I20260807 19:49:25.210795 22511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:25 dist-test-slave-btw8 krb5kdc[22497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132165, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:25.219000 22511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:25.220945 22511 webserver.cc:460] Webserver started at http://127.0.0.1:44691/ using document root <none> and password file <none>
I20260807 19:49:25.221730 22511 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:25.221880 22511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:25.222430 22511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:25.225992 22511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "3b189d01c35943a2abc79bc06841510d"
format_stamp: "Formatted at 2026-08-07 19:49:25 on dist-test-slave-btw8"
server_key: "4bc84cce366445132120f5cd210a96ff"
server_key_iv: "e78ac095dffba96969daff54939ebb83"
server_key_version: "encryptionkey@0"
I20260807 19:49:25.226912 22511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "3b189d01c35943a2abc79bc06841510d"
format_stamp: "Formatted at 2026-08-07 19:49:25 on dist-test-slave-btw8"
server_key: "4bc84cce366445132120f5cd210a96ff"
server_key_iv: "e78ac095dffba96969daff54939ebb83"
server_key_version: "encryptionkey@0"
I20260807 19:49:25.233645 22511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 19:49:25.238194 22526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:25.240268 22511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:49:25.240504 22511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "3b189d01c35943a2abc79bc06841510d"
format_stamp: "Formatted at 2026-08-07 19:49:25 on dist-test-slave-btw8"
server_key: "4bc84cce366445132120f5cd210a96ff"
server_key_iv: "e78ac095dffba96969daff54939ebb83"
server_key_version: "encryptionkey@0"
I20260807 19:49:25.240759 22511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:25.263264 22511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:25.268544 22511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:25.268957 22511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:25.291522 22581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33465 every 8 connection(s)
I20260807 19:49:25.291518 22511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33465
I20260807 19:49:25.293293 22511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:25.295193 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22511
I20260807 19:49:25.295492 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:25.296240 795 external_mini_cluster.cc:1428] Setting key 61e266e41c4e6f390b0adfe70b20bcd5
I20260807 19:49:25.298044 22582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:49:25 dist-test-slave-btw8 krb5kdc[22497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:49:25.313196 22582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d: Bootstrap starting.
I20260807 19:49:25.317736 22582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:25.319155 22582 log.cc:824] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:25.323036 22582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d: No bootstrap required, opened a new log
I20260807 19:49:25.324496 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:25.298520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33939 (local address 127.0.0.1:33465)
0807 19:49:25.299451 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:25.299467 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:25.299520 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:49:25.301626 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:25.301655 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:25.302248 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:25.302743 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:25.304323 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:25.306475 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:25.308056 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:25.309123 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:25.309349 (+ 226us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:25.314060 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:25.314079 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:25.314110 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:25.314163 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:25.318550 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:25.319696 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:25.319708 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:25.319721 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:25.320035 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:25.320656 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:25.320661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:25.320663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:25.321456 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:25.321798 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:25.322344 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 19:49:25.322710 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":380,"threads_started":1}
I20260807 19:49:25.329576 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:25.333789 22582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260807 19:49:25.334295 22582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:25.334408 22582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b189d01c35943a2abc79bc06841510d, State: Initialized, Role: FOLLOWER
I20260807 19:49:25.335601 22582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260807 19:49:25.336016 22582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:25.336215 22582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:25.336447 22582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:25.339176 22582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260807 19:49:25.339905 22582 leader_election.cc:302] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b189d01c35943a2abc79bc06841510d; no voters:
I20260807 19:49:25.340727 22582 leader_election.cc:288] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:25.340857 22592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:25.343693 22592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [term 1 LEADER]: Becoming Leader. State: Replica: 3b189d01c35943a2abc79bc06841510d, State: Running, Role: LEADER
I20260807 19:49:25.344437 22582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:25.344520 22592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260807 19:49:25.350096 22593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b189d01c35943a2abc79bc06841510d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } } }
I20260807 19:49:25.350628 22593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:25.351186 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b189d01c35943a2abc79bc06841510d. Latest consensus state: current_term: 1 leader_uuid: "3b189d01c35943a2abc79bc06841510d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b189d01c35943a2abc79bc06841510d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } } }
I20260807 19:49:25.351536 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:25.354218 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:25.335087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34929 (local address 127.0.0.1:33465)
0807 19:49:25.335339 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:25.335371 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:25.335982 (+ 611us) server_negotiation.cc:407] Connection header received
0807 19:49:25.336188 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:25.336198 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:25.336290 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:25.336440 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:25.338053 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:25.338854 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:25.340400 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:25.341010 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:25.341192 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:25.344265 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:25.344271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:25.344276 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:25.344321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:25.348374 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:25.349808 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:25.349816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:25.349822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:25.350101 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:25.351007 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:25.351014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:25.351018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:25.351541 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:25.351940 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:25.353485 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0807 19:49:25.353735 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:49:25.355002 22602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:25.360724 22602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:25.371222 22602 catalog_manager.cc:1347] Generated new cluster ID: b1638b6b4be64f0aae40895e742e1074
I20260807 19:49:25.371423 22602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:25.403777 22602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:25.405340 22602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:25.417470 22602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b189d01c35943a2abc79bc06841510d: Generated new TSK 0
I20260807 19:49:25.418493 22602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:25.433661 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22511
I20260807 19:49:25.435767 22577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:25.536430 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22511
W20260807 19:49:25.558355 22495 connection.cc:535] client connection to 127.0.0.1:33465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33465): Connection reset by peer (error 104)
2026-08-07T19:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:25 dist-test-slave-btw8 krb5kdc[22614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:25 dist-test-slave-btw8 krb5kdc[22614](info): set up 2 sockets
Aug 07 19:49:25 dist-test-slave-btw8 krb5kdc[22614](info): commencing operation
krb5kdc: starting...
W20260807 19:49:26.760753 795 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:27 dist-test-slave-btw8 krb5kdc[22614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:27.156658 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:27.542380 22629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:27.542745 22629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:27.542924 22629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:27.550572 22629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:27.550714 22629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:27.550761 22629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:27.550894 22629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:27.550987 22629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:27.561573 22629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:27.563112 22629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:27.564646 22629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:27.578552 22637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:27.578590 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:27.578500 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:27.579955 22629 server_base.cc:1034] running on GCE node
I20260807 19:49:27.581141 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:27.582877 22629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:27.584156 22629 hybrid_clock.cc:648] HybridClock initialized: now 1786132167584103 us; error 47 us; skew 500 ppm
I20260807 19:49:27.584514 22629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:27 dist-test-slave-btw8 krb5kdc[22614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:27.588264 22629 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:27.590698 22629 webserver.cc:460] Webserver started at http://127.0.0.1:35209/ using document root <none> and password file <none>
I20260807 19:49:27.591679 22629 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:27.591858 22629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:27.592247 22629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:27.595968 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "fe3caca75320445987a07e1f158e6500"
format_stamp: "Formatted at 2026-08-07 19:49:27 on dist-test-slave-btw8"
server_key: "02210524078da18d7b8dddae6b698b8d"
server_key_iv: "60af2a3ecf2b0a2937374b822890a450"
server_key_version: "encryptionkey@0"
I20260807 19:49:27.596939 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "fe3caca75320445987a07e1f158e6500"
format_stamp: "Formatted at 2026-08-07 19:49:27 on dist-test-slave-btw8"
server_key: "02210524078da18d7b8dddae6b698b8d"
server_key_iv: "60af2a3ecf2b0a2937374b822890a450"
server_key_version: "encryptionkey@0"
I20260807 19:49:27.603529 22629 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:49:27.607865 22644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:27.610006 22629 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:49:27.610244 22629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "fe3caca75320445987a07e1f158e6500"
format_stamp: "Formatted at 2026-08-07 19:49:27 on dist-test-slave-btw8"
server_key: "02210524078da18d7b8dddae6b698b8d"
server_key_iv: "60af2a3ecf2b0a2937374b822890a450"
server_key_version: "encryptionkey@0"
I20260807 19:49:27.610513 22629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:27.632141 22629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:27.637661 22629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:27.638064 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:27.660632 22629 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34593
I20260807 19:49:27.660748 22696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34593 every 8 connection(s)
I20260807 19:49:27.662333 22629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:27.666854 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:27.667734 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22629
I20260807 19:49:27.667977 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:27.668610 795 external_mini_cluster.cc:1428] Setting key 280b2f0e2da78ba751a7f7844143a1a7
I20260807 19:49:27.684309 22697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500: Bootstrap starting.
I20260807 19:49:27.686187 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:27.670961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55543 (local address 127.0.0.1:34593)
0807 19:49:27.672065 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:27.672079 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:27.672132 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:49:27.674562 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:27.674606 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:27.675163 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:27.675680 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:27.676479 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:27.678080 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:27.680537 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:27.681504 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:27.681681 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:27.683320 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:27.683330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:27.683423 (+ 93us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:27.684100 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:27.684200 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:27.684230 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:27.684388 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:27.684845 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 19:49:27.685087 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":588,"threads_started":1}
I20260807 19:49:27.688584 22697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:27.690099 22697 log.cc:824] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:27.691395 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:27.693994 22697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500: No bootstrap required, opened a new log
I20260807 19:49:27.704234 22697 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260807 19:49:27.704866 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:27.696960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57501 (local address 127.0.0.1:34593)
0807 19:49:27.697193 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:27.697203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:27.697636 (+ 433us) server_negotiation.cc:407] Connection header received
0807 19:49:27.697902 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:27.697910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:27.697984 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:27.698123 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:27.698734 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:27.699740 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:27.701541 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:27.702107 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:27.702297 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:27.703835 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:27.703841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:27.703845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:27.703869 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:27.703936 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:27.703955 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:27.704124 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:27.704356 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 19:49:27.704425 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:49:27.704877 22697 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:27.705127 22697 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3caca75320445987a07e1f158e6500, State: Initialized, Role: FOLLOWER
I20260807 19:49:27.705864 22697 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260807 19:49:27.706209 22697 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:27.706387 22697 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:27.706616 22697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:27.709437 22697 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260807 19:49:27.710003 22697 leader_election.cc:302] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe3caca75320445987a07e1f158e6500; no voters:
I20260807 19:49:27.710755 22697 leader_election.cc:288] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:27.710986 22708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:27.712447 22708 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [term 1 LEADER]: Becoming Leader. State: Replica: fe3caca75320445987a07e1f158e6500, State: Running, Role: LEADER
I20260807 19:49:27.713161 22708 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260807 19:49:27.713462 22697 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:27.717024 22710 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe3caca75320445987a07e1f158e6500. Latest consensus state: current_term: 1 leader_uuid: "fe3caca75320445987a07e1f158e6500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } } }
I20260807 19:49:27.717526 22710 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:27.718137 22709 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe3caca75320445987a07e1f158e6500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3caca75320445987a07e1f158e6500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } } }
I20260807 19:49:27.718667 22709 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:27.721410 22716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:27.727030 22716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:27.737804 22716 catalog_manager.cc:1347] Generated new cluster ID: 6e87aa2fa3b746d1bcb8c057e2715b8f
I20260807 19:49:27.737954 22716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:27.777863 22716 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:27.779984 22716 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:27.789585 22716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe3caca75320445987a07e1f158e6500: Generated new TSK 0
I20260807 19:49:27.790784 22716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:27.801278 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22629
W20260807 19:49:27.802609 22692 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55543
W20260807 19:49:27.803632 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22629: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:27.803822 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22629
W20260807 19:49:27.826572 22612 connection.cc:535] client connection to 127.0.0.1:34593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34593): Connection reset by peer (error 104)
2026-08-07T19:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:27 dist-test-slave-btw8 krb5kdc[22730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:27 dist-test-slave-btw8 krb5kdc[22730](info): set up 2 sockets
Aug 07 19:49:27 dist-test-slave-btw8 krb5kdc[22730](info): commencing operation
krb5kdc: starting...
W20260807 19:49:29.060755 795 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:29 dist-test-slave-btw8 krb5kdc[22730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:29.443001 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:29.835139 22743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:29.835587 22743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:29.835772 22743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:29.844241 22743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:29.844410 22743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:29.844501 22743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:29.844650 22743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:29.844730 22743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:29.856637 22743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:29.858685 22743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:29.860442 22743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:29.874927 22749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:29.874879 22750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:29.874969 22752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:29.876339 22743 server_base.cc:1034] running on GCE node
I20260807 19:49:29.877305 22743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:29.879256 22743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:29.880676 22743 hybrid_clock.cc:648] HybridClock initialized: now 1786132169880592 us; error 75 us; skew 500 ppm
I20260807 19:49:29.881160 22743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:29 dist-test-slave-btw8 krb5kdc[22730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:29.885066 22743 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:29.886968 22743 webserver.cc:460] Webserver started at http://127.0.0.1:44047/ using document root <none> and password file <none>
I20260807 19:49:29.887876 22743 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:29.888048 22743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:29.888419 22743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:29.892220 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "fdc7afd0fb4b48c2b3034489ea80d87c"
format_stamp: "Formatted at 2026-08-07 19:49:29 on dist-test-slave-btw8"
server_key: "eb3b12de41807086075b52dbce837ab2"
server_key_iv: "a1ceef597f66c6b5dc75238bb9b8b607"
server_key_version: "encryptionkey@0"
I20260807 19:49:29.893185 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "fdc7afd0fb4b48c2b3034489ea80d87c"
format_stamp: "Formatted at 2026-08-07 19:49:29 on dist-test-slave-btw8"
server_key: "eb3b12de41807086075b52dbce837ab2"
server_key_iv: "a1ceef597f66c6b5dc75238bb9b8b607"
server_key_version: "encryptionkey@0"
I20260807 19:49:29.899618 22743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260807 19:49:29.905821 22760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:29.908743 22743 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 19:49:29.909040 22743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "fdc7afd0fb4b48c2b3034489ea80d87c"
format_stamp: "Formatted at 2026-08-07 19:49:29 on dist-test-slave-btw8"
server_key: "eb3b12de41807086075b52dbce837ab2"
server_key_iv: "a1ceef597f66c6b5dc75238bb9b8b607"
server_key_version: "encryptionkey@0"
I20260807 19:49:29.909386 22743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:29.919214 22743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:29.924748 22743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:29.925076 22743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:29.947731 22743 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35687
I20260807 19:49:29.947803 22812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35687 every 8 connection(s)
I20260807 19:49:29.949414 22743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:29.951306 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22743
I20260807 19:49:29.951550 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:29.952131 795 external_mini_cluster.cc:1428] Setting key c11138f46baa5aac2d7178f1e4a95098
I20260807 19:49:29.954536 22813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:29.961122 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:29.954146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38743 (local address 127.0.0.1:35687)
0807 19:49:29.955092 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:29.955104 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:29.955151 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:49:29.957040 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:29.957066 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:29.957461 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:29.957861 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:29.959175 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:29.959181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:29.959196 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:29.959214 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:29.959313 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:29.959336 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:29.959599 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:29.959788 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 19:49:29.960061 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":478,"threads_started":1}
I20260807 19:49:29.965803 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:29.968297 22813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c: Bootstrap starting.
I20260807 19:49:29.972391 22813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:29.974020 22813 log.cc:824] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:29.974288 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:29.971469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45027 (local address 127.0.0.1:35687)
0807 19:49:29.971717 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:29.971723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:29.972091 (+ 368us) server_negotiation.cc:407] Connection header received
0807 19:49:29.972279 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:29.972283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:29.972341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:29.972440 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:29.973535 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:29.973539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:29.973543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:29.973560 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:29.973610 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:29.973628 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:29.973785 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:29.973891 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 19:49:29.973946 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:49:29.978236 22813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c: No bootstrap required, opened a new log
I20260807 19:49:29.986804 22813 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } }
I20260807 19:49:29.987252 22813 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:29.987532 22813 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc7afd0fb4b48c2b3034489ea80d87c, State: Initialized, Role: FOLLOWER
I20260807 19:49:29.988579 22813 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } }
I20260807 19:49:29.988837 22813 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:29.988940 22813 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:29.989084 22813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:29.991545 22813 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } }
I20260807 19:49:29.992178 22813 leader_election.cc:302] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc7afd0fb4b48c2b3034489ea80d87c; no voters:
I20260807 19:49:29.992975 22813 leader_election.cc:288] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:29.993152 22824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:29.994025 22824 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [term 1 LEADER]: Becoming Leader. State: Replica: fdc7afd0fb4b48c2b3034489ea80d87c, State: Running, Role: LEADER
I20260807 19:49:29.994637 22824 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } }
I20260807 19:49:29.995476 22813 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:29.997912 22825 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } } }
I20260807 19:49:29.997997 22826 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc7afd0fb4b48c2b3034489ea80d87c. Latest consensus state: current_term: 1 leader_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc7afd0fb4b48c2b3034489ea80d87c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35687 } } }
I20260807 19:49:29.998623 22826 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:29.998960 22825 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:30.003856 22832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:30.009744 22832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:30.020922 22832 catalog_manager.cc:1347] Generated new cluster ID: 6ff97dc6489e4e559486f8390b7fe5e4
I20260807 19:49:30.021070 22832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:30.051855 22832 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:30.052407 22832 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:30.072187 22832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc7afd0fb4b48c2b3034489ea80d87c: Generated new TSK 0
I20260807 19:49:30.073187 22832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:30.091099 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22743
W20260807 19:49:30.092551 22808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38743
W20260807 19:49:30.093464 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22743: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:30.093645 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22743
2026-08-07T19:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:30 dist-test-slave-btw8 krb5kdc[22846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:30 dist-test-slave-btw8 krb5kdc[22846](info): set up 2 sockets
Aug 07 19:49:30 dist-test-slave-btw8 krb5kdc[22846](info): commencing operation
krb5kdc: starting...
W20260807 19:49:31.306586 795 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.004s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:31 dist-test-slave-btw8 krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:31.692592 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:32.079085 22859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:32.079443 22859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:32.079545 22859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:32.087291 22859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:32.087476 22859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:32.087528 22859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:32.087606 22859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:32.087646 22859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:32.098582 22859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:32.100111 22859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:32.101672 22859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:32.115880 22868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:32.116159 22865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:32.115948 22866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:32.116925 22859 server_base.cc:1034] running on GCE node
I20260807 19:49:32.117905 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:32.119701 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:32.120977 22859 hybrid_clock.cc:648] HybridClock initialized: now 1786132172120927 us; error 45 us; skew 500 ppm
I20260807 19:49:32.121392 22859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:32 dist-test-slave-btw8 krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:32.130944 22859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:32.133145 22859 webserver.cc:460] Webserver started at http://127.0.0.1:40871/ using document root <none> and password file <none>
I20260807 19:49:32.134048 22859 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:32.134223 22859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:32.134575 22859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:32.138137 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "d75f5734ffdd41d68ea554c37ee20f6c"
format_stamp: "Formatted at 2026-08-07 19:49:32 on dist-test-slave-btw8"
server_key: "13559c76870059ba5cf2b03b26677527"
server_key_iv: "6fdcfc5da95411395dee530377f41ec3"
server_key_version: "encryptionkey@0"
I20260807 19:49:32.139139 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "d75f5734ffdd41d68ea554c37ee20f6c"
format_stamp: "Formatted at 2026-08-07 19:49:32 on dist-test-slave-btw8"
server_key: "13559c76870059ba5cf2b03b26677527"
server_key_iv: "6fdcfc5da95411395dee530377f41ec3"
server_key_version: "encryptionkey@0"
I20260807 19:49:32.146330 22859 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260807 19:49:32.151834 22876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:32.154193 22859 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:49:32.154498 22859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "d75f5734ffdd41d68ea554c37ee20f6c"
format_stamp: "Formatted at 2026-08-07 19:49:32 on dist-test-slave-btw8"
server_key: "13559c76870059ba5cf2b03b26677527"
server_key_iv: "6fdcfc5da95411395dee530377f41ec3"
server_key_version: "encryptionkey@0"
I20260807 19:49:32.154771 22859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:32.175995 22859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:32.182368 22859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:32.182773 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:32.204530 22859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36307
I20260807 19:49:32.204568 22932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36307 every 8 connection(s)
I20260807 19:49:32.206089 22859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:32.210515 22933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:32.211594 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22859
I20260807 19:49:32.211939 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:32.212534 795 external_mini_cluster.cc:1428] Setting key 397fb65cad2a739076d89a110c4d5f0d
I20260807 19:49:32.225046 22933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c: Bootstrap starting.
Aug 07 19:49:32 dist-test-slave-btw8 krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:49:32.228820 22933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:32.230036 22933 log.cc:824] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:32.233605 22933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c: No bootstrap required, opened a new log
I20260807 19:49:32.237828 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:32.214688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42775 (local address 127.0.0.1:36307)
0807 19:49:32.215742 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:32.215757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:32.215808 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:49:32.218042 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:32.218077 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:32.218598 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:32.219057 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:32.220639 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:32.221993 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:32.223645 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:32.224381 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:32.224599 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:32.229422 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:32.229432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:32.229448 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:32.229491 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:32.233008 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:32.233956 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:32.233963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:32.233970 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:32.234244 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:32.234663 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:32.234674 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:32.234677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:32.235301 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:32.235720 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:32.236181 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:49:32.236693 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":404,"threads_started":1}
I20260807 19:49:32.242200 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:32.242985 22933 raft_consensus.cc:348] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } }
I20260807 19:49:32.243412 22933 raft_consensus.cc:374] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:32.243517 22933 raft_consensus.cc:729] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75f5734ffdd41d68ea554c37ee20f6c, State: Initialized, Role: FOLLOWER
I20260807 19:49:32.244187 22933 consensus_queue.cc:260] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } }
I20260807 19:49:32.244439 22933 raft_consensus.cc:388] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:32.244596 22933 raft_consensus.cc:482] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:32.244773 22933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:32.247534 22933 raft_consensus.cc:504] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } }
I20260807 19:49:32.248164 22933 leader_election.cc:302] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d75f5734ffdd41d68ea554c37ee20f6c; no voters:
I20260807 19:49:32.248831 22933 leader_election.cc:288] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:32.249013 22943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:32.249763 22943 raft_consensus.cc:686] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [term 1 LEADER]: Becoming Leader. State: Replica: d75f5734ffdd41d68ea554c37ee20f6c, State: Running, Role: LEADER
I20260807 19:49:32.250499 22943 consensus_queue.cc:237] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } }
I20260807 19:49:32.252418 22943 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d75f5734ffdd41d68ea554c37ee20f6c. Latest consensus state: current_term: 1 leader_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } } }
I20260807 19:49:32.252821 22943 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:32.257956 22945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75f5734ffdd41d68ea554c37ee20f6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36307 } } }
I20260807 19:49:32.258345 22945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:32.258365 22933 sys_catalog.cc:564] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:32.258669 22947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:32.270623 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:32.247096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42601 (local address 127.0.0.1:36307)
0807 19:49:32.247312 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:32.247322 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:32.247706 (+ 384us) server_negotiation.cc:407] Connection header received
0807 19:49:32.247920 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:32.247925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:32.248009 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:32.248124 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:32.250330 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:32.251898 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:32.254873 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:32.255793 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:32.255960 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:32.258850 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:32.258858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:32.258864 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:32.258909 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:32.263973 (+ 5064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:32.265436 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:32.265443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:32.265450 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:32.265706 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:32.266370 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:32.266382 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:32.266388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:32.267039 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:32.267420 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:32.268779 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0807 19:49:32.269625 (+ 846us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:49:32.273485 22947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:32.290678 22947 catalog_manager.cc:1347] Generated new cluster ID: 4f27bd20d47e4c90a192243887fb6405
I20260807 19:49:32.291064 22947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:32.334291 22947 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:32.335884 22947 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:32.360042 22947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d75f5734ffdd41d68ea554c37ee20f6c: Generated new TSK 0
I20260807 19:49:32.361205 22947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:32.372783 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22859
I20260807 19:49:32.373965 22928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:32.475860 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22859
2026-08-07T19:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:32 dist-test-slave-btw8 krb5kdc[22968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:32 dist-test-slave-btw8 krb5kdc[22968](info): set up 2 sockets
Aug 07 19:49:32 dist-test-slave-btw8 krb5kdc[22968](info): commencing operation
krb5kdc: starting...
W20260807 19:49:33.728587 795 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.006s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:33 dist-test-slave-btw8 krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:34.124047 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:34.505776 22981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:34.506178 22981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:34.506366 22981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:34.513974 22981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:34.514170 22981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:34.514292 22981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:34.514410 22981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:34.514508 22981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:34.525525 22981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:34.527036 22981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:34.528505 22981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:34.541829 22987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:34.541846 22990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:34.541862 22988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:34.542870 22981 server_base.cc:1034] running on GCE node
I20260807 19:49:34.543871 22981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:34.545424 22981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:34.546772 22981 hybrid_clock.cc:648] HybridClock initialized: now 1786132174546689 us; error 67 us; skew 500 ppm
I20260807 19:49:34.547211 22981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:34 dist-test-slave-btw8 krb5kdc[22968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:34.550968 22981 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:34.553009 22981 webserver.cc:460] Webserver started at http://127.0.0.1:39109/ using document root <none> and password file <none>
I20260807 19:49:34.553764 22981 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:34.553890 22981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:34.554188 22981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:34.557713 22981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "b66da35ea75b45aca0a3b96f3baa9536"
format_stamp: "Formatted at 2026-08-07 19:49:34 on dist-test-slave-btw8"
server_key: "e1ab7c68c4ff06bf3e8b3084aac7d3f3"
server_key_iv: "5e0bea6212ec303deeaa3037a13bc0c3"
server_key_version: "encryptionkey@0"
I20260807 19:49:34.558699 22981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "b66da35ea75b45aca0a3b96f3baa9536"
format_stamp: "Formatted at 2026-08-07 19:49:34 on dist-test-slave-btw8"
server_key: "e1ab7c68c4ff06bf3e8b3084aac7d3f3"
server_key_iv: "5e0bea6212ec303deeaa3037a13bc0c3"
server_key_version: "encryptionkey@0"
I20260807 19:49:34.565057 22981 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:49:34.569314 22997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:34.571218 22981 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260807 19:49:34.571499 22981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "b66da35ea75b45aca0a3b96f3baa9536"
format_stamp: "Formatted at 2026-08-07 19:49:34 on dist-test-slave-btw8"
server_key: "e1ab7c68c4ff06bf3e8b3084aac7d3f3"
server_key_iv: "5e0bea6212ec303deeaa3037a13bc0c3"
server_key_version: "encryptionkey@0"
I20260807 19:49:34.571859 22981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:34.601217 22981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:34.607013 22981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:34.607467 22981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:34.629843 22981 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45213
I20260807 19:49:34.629884 23052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45213 every 8 connection(s)
I20260807 19:49:34.631738 22981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:34.633424 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 22981
I20260807 19:49:34.633657 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:34.634140 795 external_mini_cluster.cc:1428] Setting key cb815642eed52c9514a11aae80edf9d9
I20260807 19:49:34.637017 23053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:34.651472 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:34.636269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58919 (local address 127.0.0.1:45213)
0807 19:49:34.637244 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:34.637261 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:34.637316 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:49:34.639862 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:34.639895 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:34.640493 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:34.640977 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:34.641561 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:34.643742 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:34.645453 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:34.646561 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:34.646772 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:34.648184 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:34.648195 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:34.648228 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:34.649102 (+ 874us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:34.649229 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:34.649266 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:34.649402 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:34.649665 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 19:49:34.649999 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":380,"threads_started":1}
I20260807 19:49:34.652549 23053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536: Bootstrap starting.
I20260807 19:49:34.656196 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:34.656219 23053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:34.657291 23053 log.cc:824] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:34.660934 23053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536: No bootstrap required, opened a new log
I20260807 19:49:34.669850 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:34.661878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40133 (local address 127.0.0.1:45213)
0807 19:49:34.662065 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:34.662071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:34.662561 (+ 490us) server_negotiation.cc:407] Connection header received
0807 19:49:34.662736 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:34.662745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:34.662840 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:34.662961 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:34.663717 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:34.664919 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:34.666563 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:34.667388 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:34.667538 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:34.668923 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:34.668930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:34.668935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:34.668962 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:34.669018 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:34.669042 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:34.669149 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:34.669332 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:49:34.669461 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:49:34.671468 23053 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } }
I20260807 19:49:34.672019 23053 raft_consensus.cc:374] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:34.672151 23053 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66da35ea75b45aca0a3b96f3baa9536, State: Initialized, Role: FOLLOWER
I20260807 19:49:34.672788 23053 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } }
I20260807 19:49:34.673064 23053 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:34.673225 23053 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:34.673409 23053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:34.676108 23053 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } }
I20260807 19:49:34.676715 23053 leader_election.cc:302] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66da35ea75b45aca0a3b96f3baa9536; no voters:
I20260807 19:49:34.677502 23053 leader_election.cc:288] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:34.677618 23063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:34.679315 23063 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [term 1 LEADER]: Becoming Leader. State: Replica: b66da35ea75b45aca0a3b96f3baa9536, State: Running, Role: LEADER
I20260807 19:49:34.680094 23063 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } }
I20260807 19:49:34.680341 23053 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:34.684055 23064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b66da35ea75b45aca0a3b96f3baa9536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } } }
I20260807 19:49:34.684703 23064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:34.685554 23065 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66da35ea75b45aca0a3b96f3baa9536. Latest consensus state: current_term: 1 leader_uuid: "b66da35ea75b45aca0a3b96f3baa9536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66da35ea75b45aca0a3b96f3baa9536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45213 } } }
I20260807 19:49:34.685887 23071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:34.685983 23065 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:34.691929 23071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:34.702785 23071 catalog_manager.cc:1347] Generated new cluster ID: 6c5966ea7f794f3c838cbae61680ae2e
I20260807 19:49:34.702996 23071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:34.748193 23071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:34.749927 23071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:34.777859 23071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b66da35ea75b45aca0a3b96f3baa9536: Generated new TSK 0
I20260807 19:49:34.779131 23071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:34.801967 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22981
W20260807 19:49:34.803326 23047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58919
W20260807 19:49:34.804212 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 22981: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:34.804388 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 22981
2026-08-07T19:49:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:34 dist-test-slave-btw8 krb5kdc[23084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:34 dist-test-slave-btw8 krb5kdc[23084](info): set up 2 sockets
Aug 07 19:49:34 dist-test-slave-btw8 krb5kdc[23084](info): commencing operation
krb5kdc: starting...
W20260807 19:49:36.050053 795 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:36 dist-test-slave-btw8 krb5kdc[23084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:36.439323 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:36.830606 23099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:36.830997 23099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:36.831305 23099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:36.839457 23099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:36.839617 23099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:36.839676 23099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:36.839789 23099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:36.839859 23099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:36.850818 23099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:36.852382 23099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:36.853984 23099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:36.868345 23104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:36.868456 23105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:36.868958 23107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:36.870370 23099 server_base.cc:1034] running on GCE node
I20260807 19:49:36.871727 23099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:36.873692 23099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:36.875020 23099 hybrid_clock.cc:648] HybridClock initialized: now 1786132176874944 us; error 85 us; skew 500 ppm
I20260807 19:49:36.875591 23099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:36 dist-test-slave-btw8 krb5kdc[23084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:36.880247 23099 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:36.882807 23099 webserver.cc:460] Webserver started at http://127.0.0.1:33475/ using document root <none> and password file <none>
I20260807 19:49:36.883749 23099 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:36.883929 23099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:36.884294 23099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:36.888126 23099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "4d81e880860a4fcbaafaa569bcf77307"
format_stamp: "Formatted at 2026-08-07 19:49:36 on dist-test-slave-btw8"
server_key: "1789e457bb95bfd59af0539692a839b8"
server_key_iv: "fab78c28de80eec32f2f556d61df65b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:36.889146 23099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "4d81e880860a4fcbaafaa569bcf77307"
format_stamp: "Formatted at 2026-08-07 19:49:36 on dist-test-slave-btw8"
server_key: "1789e457bb95bfd59af0539692a839b8"
server_key_iv: "fab78c28de80eec32f2f556d61df65b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:36.897182 23099 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260807 19:49:36.902582 23115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:36.904759 23099 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 19:49:36.905019 23099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "4d81e880860a4fcbaafaa569bcf77307"
format_stamp: "Formatted at 2026-08-07 19:49:36 on dist-test-slave-btw8"
server_key: "1789e457bb95bfd59af0539692a839b8"
server_key_iv: "fab78c28de80eec32f2f556d61df65b1"
server_key_version: "encryptionkey@0"
I20260807 19:49:36.905306 23099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:36.913637 23099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:36.919803 23099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:36.920202 23099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:36.945503 23099 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46389
I20260807 19:49:36.945616 23168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46389 every 8 connection(s)
I20260807 19:49:36.947203 23099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:36.948971 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23099
I20260807 19:49:36.949167 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:36.949752 795 external_mini_cluster.cc:1428] Setting key 3da3ce7d91bf95ffb0da79bcb8821392
I20260807 19:49:36.952843 23169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:36.960225 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:36.952079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34961 (local address 127.0.0.1:46389)
0807 19:49:36.953398 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:36.953426 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:36.953483 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:49:36.955651 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:36.955681 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:36.956139 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:36.956685 (+ 546us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:36.958062 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:36.958069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:36.958085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:36.958108 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:36.958204 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:36.958230 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:36.958598 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:36.958781 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:49:36.959027 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":441,"threads_started":1}
I20260807 19:49:36.966017 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:36.967819 23169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307: Bootstrap starting.
I20260807 19:49:36.972906 23169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:36.974484 23169 log.cc:824] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:36.975466 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:36.972098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46845 (local address 127.0.0.1:46389)
0807 19:49:36.972458 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:36.972464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:36.972880 (+ 416us) server_negotiation.cc:407] Connection header received
0807 19:49:36.973070 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:36.973076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:36.973147 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:36.973285 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:36.974391 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:36.974397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:36.974402 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:36.974425 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:36.974486 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:36.974510 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:36.974663 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:36.974940 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:49:36.975008 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260807 19:49:36.978914 23169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307: No bootstrap required, opened a new log
I20260807 19:49:36.988628 23169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } }
I20260807 19:49:36.989099 23169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:36.989215 23169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d81e880860a4fcbaafaa569bcf77307, State: Initialized, Role: FOLLOWER
I20260807 19:49:36.989986 23169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } }
I20260807 19:49:36.990265 23169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:36.990428 23169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:36.990618 23169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:36.993216 23169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } }
I20260807 19:49:36.994482 23169 leader_election.cc:302] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d81e880860a4fcbaafaa569bcf77307; no voters:
I20260807 19:49:36.995555 23169 leader_election.cc:288] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:36.995853 23180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:36.996869 23180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [term 1 LEADER]: Becoming Leader. State: Replica: 4d81e880860a4fcbaafaa569bcf77307, State: Running, Role: LEADER
I20260807 19:49:36.997720 23180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } }
I20260807 19:49:36.998349 23169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:37.001611 23181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d81e880860a4fcbaafaa569bcf77307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } } }
I20260807 19:49:37.002458 23181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:37.001816 23182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d81e880860a4fcbaafaa569bcf77307. Latest consensus state: current_term: 1 leader_uuid: "4d81e880860a4fcbaafaa569bcf77307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d81e880860a4fcbaafaa569bcf77307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46389 } } }
I20260807 19:49:37.003260 23182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:37.003631 23188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:37.010366 23188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:37.029806 23188 catalog_manager.cc:1347] Generated new cluster ID: ffd8c4106599437286185e99bed203df
I20260807 19:49:37.029989 23188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:37.074537 23188 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:37.075222 23188 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:37.086437 23188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d81e880860a4fcbaafaa569bcf77307: Generated new TSK 0
I20260807 19:49:37.087759 23188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:37.099345 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23099
W20260807 19:49:37.100759 23164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34961
W20260807 19:49:37.101612 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23099: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:37.101817 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23099
2026-08-07T19:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:37 dist-test-slave-btw8 krb5kdc[23203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:37 dist-test-slave-btw8 krb5kdc[23203](info): set up 2 sockets
Aug 07 19:49:37 dist-test-slave-btw8 krb5kdc[23203](info): commencing operation
krb5kdc: starting...
W20260807 19:49:38.388793 795 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.004s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:38 dist-test-slave-btw8 krb5kdc[23203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:38.792984 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40397
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:39.206393 23216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:39.206786 23216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:39.207026 23216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:39.215807 23216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:39.215968 23216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:39.216087 23216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:39.216192 23216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:39.216269 23216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:39.228615 23216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:39.230217 23216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:39.232080 23216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:39.249233 23224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:39.249352 23221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:39.249575 23222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:39.250931 23216 server_base.cc:1034] running on GCE node
I20260807 19:49:39.252049 23216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:39.254220 23216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:39.255569 23216 hybrid_clock.cc:648] HybridClock initialized: now 1786132179255494 us; error 49 us; skew 500 ppm
I20260807 19:49:39.256062 23216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:39 dist-test-slave-btw8 krb5kdc[23203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:39.260468 23216 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:39.262723 23216 webserver.cc:460] Webserver started at http://127.0.0.1:41857/ using document root <none> and password file <none>
I20260807 19:49:39.263794 23216 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:39.264000 23216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:39.264542 23216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:39.268620 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "be574a6f73524efd8ca487549347e704"
format_stamp: "Formatted at 2026-08-07 19:49:39 on dist-test-slave-btw8"
server_key: "9bf98bc903c75a30e60018c80654ac0b"
server_key_iv: "edc5d1b2391f3358d734025702d75df9"
server_key_version: "encryptionkey@0"
I20260807 19:49:39.269711 23216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "be574a6f73524efd8ca487549347e704"
format_stamp: "Formatted at 2026-08-07 19:49:39 on dist-test-slave-btw8"
server_key: "9bf98bc903c75a30e60018c80654ac0b"
server_key_iv: "edc5d1b2391f3358d734025702d75df9"
server_key_version: "encryptionkey@0"
I20260807 19:49:39.278229 23216 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260807 19:49:39.283667 23234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:39.285626 23216 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:49:39.285921 23216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "be574a6f73524efd8ca487549347e704"
format_stamp: "Formatted at 2026-08-07 19:49:39 on dist-test-slave-btw8"
server_key: "9bf98bc903c75a30e60018c80654ac0b"
server_key_iv: "edc5d1b2391f3358d734025702d75df9"
server_key_version: "encryptionkey@0"
I20260807 19:49:39.286195 23216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:39.309170 23216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:39.315260 23216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:39.315694 23216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:39.342866 23216 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40397
I20260807 19:49:39.342905 23289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40397 every 8 connection(s)
I20260807 19:49:39.344641 23216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:39.345628 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23216
I20260807 19:49:39.345891 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:39.346400 795 external_mini_cluster.cc:1428] Setting key b1d3a1e329ed701acc2a32e22c7e8621
I20260807 19:49:39.350610 23290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:49:39 dist-test-slave-btw8 krb5kdc[23203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:49:39.367121 23290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704: Bootstrap starting.
I20260807 19:49:39.372331 23290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:39.373728 23290 log.cc:824] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:39.378327 23290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704: No bootstrap required, opened a new log
I20260807 19:49:39.378979 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:39.348760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:45553 (local address 127.0.0.1:40397)
0807 19:49:39.349793 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:39.349811 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:39.349872 (+ 61us) server_negotiation.cc:407] Connection header received
0807 19:49:39.352764 (+ 2892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:39.352901 (+ 137us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:39.353595 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:39.354174 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:39.355939 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:39.358278 (+ 2339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:39.360809 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:39.362004 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:39.362253 (+ 249us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:39.367886 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:39.367898 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:39.367922 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:39.367979 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:39.372667 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:39.374003 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:39.374020 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:39.374029 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:39.374299 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:39.375046 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:39.375050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:39.375052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:39.375991 (+ 939us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:39.376348 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:39.376975 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 19:49:39.377398 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":485,"threads_started":1}
I20260807 19:49:39.385174 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:39.390350 23290 raft_consensus.cc:348] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } }
I20260807 19:49:39.391036 23290 raft_consensus.cc:374] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:39.391185 23290 raft_consensus.cc:729] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be574a6f73524efd8ca487549347e704, State: Initialized, Role: FOLLOWER
I20260807 19:49:39.392436 23290 consensus_queue.cc:260] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } }
I20260807 19:49:39.393005 23290 raft_consensus.cc:388] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:39.393200 23290 raft_consensus.cc:482] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:39.393429 23290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:39.396337 23290 raft_consensus.cc:504] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } }
I20260807 19:49:39.397025 23290 leader_election.cc:302] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be574a6f73524efd8ca487549347e704; no voters:
I20260807 19:49:39.397883 23290 leader_election.cc:288] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:39.398034 23300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:39.401229 23290 sys_catalog.cc:564] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:39.405223 23300 raft_consensus.cc:686] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [term 1 LEADER]: Becoming Leader. State: Replica: be574a6f73524efd8ca487549347e704, State: Running, Role: LEADER
I20260807 19:49:39.406438 23300 consensus_queue.cc:237] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } }
I20260807 19:49:39.411748 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:39.392634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:57069 (local address 127.0.0.1:40397)
0807 19:49:39.392879 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:39.392886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:39.393342 (+ 456us) server_negotiation.cc:407] Connection header received
0807 19:49:39.393629 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:39.393633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:39.393704 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:39.393880 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:39.395434 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:39.396227 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:39.397747 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:39.398415 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:39.399329 (+ 914us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:39.401979 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:39.401992 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:39.401999 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:39.402061 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:39.407047 (+ 4986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:39.408208 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:39.408215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:39.408219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:39.408412 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:39.409520 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:39.409526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:39.409529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:39.410079 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:39.410496 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:39.410993 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 19:49:39.411212 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:49:39.420679 23301 sys_catalog.cc:455] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be574a6f73524efd8ca487549347e704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } } }
I20260807 19:49:39.424613 23301 sys_catalog.cc:458] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:39.425366 23302 sys_catalog.cc:455] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be574a6f73524efd8ca487549347e704. Latest consensus state: current_term: 1 leader_uuid: "be574a6f73524efd8ca487549347e704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be574a6f73524efd8ca487549347e704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40397 } } }
I20260807 19:49:39.425710 23302 sys_catalog.cc:458] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:39.425865 23314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:39.433061 23314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:39.447465 23314 catalog_manager.cc:1347] Generated new cluster ID: c999e27c4bb84b289dd10df378b42f9b
I20260807 19:49:39.447626 23314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:39.463195 23314 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:39.464884 23314 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:39.480098 23314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be574a6f73524efd8ca487549347e704: Generated new TSK 0
I20260807 19:49:39.481575 23314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:39.490965 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23216
I20260807 19:49:39.493144 23285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:39.592337 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23216
W20260807 19:49:39.615871 23200 connection.cc:535] client connection to 127.0.0.1:40397 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40397): Connection reset by peer (error 104)
2026-08-07T19:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:39 dist-test-slave-btw8 krb5kdc[23323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:39 dist-test-slave-btw8 krb5kdc[23323](info): set up 2 sockets
Aug 07 19:49:39 dist-test-slave-btw8 krb5kdc[23323](info): commencing operation
krb5kdc: starting...
W20260807 19:49:40.883723 795 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:41 dist-test-slave-btw8 krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:41.292363 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:41.707789 23336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:41.708144 23336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:41.708314 23336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:41.716058 23336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:41.716234 23336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:41.716355 23336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:41.716445 23336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:41.716522 23336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:41.727257 23336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:41.728809 23336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:41.730269 23336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:41.743753 23342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:41.743700 23343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:41.743770 23345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:41.745309 23336 server_base.cc:1034] running on GCE node
I20260807 19:49:41.746263 23336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:41.748008 23336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:41.749281 23336 hybrid_clock.cc:648] HybridClock initialized: now 1786132181749217 us; error 42 us; skew 500 ppm
I20260807 19:49:41.749711 23336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:41 dist-test-slave-btw8 krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:41.753328 23336 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:41.755256 23336 webserver.cc:460] Webserver started at http://127.0.0.1:42407/ using document root <none> and password file <none>
I20260807 19:49:41.756054 23336 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:41.756197 23336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:41.756554 23336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:41.759999 23336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "8eee39f8ccf648cf9861460300596906"
format_stamp: "Formatted at 2026-08-07 19:49:41 on dist-test-slave-btw8"
server_key: "9ee4615e1e8f56c7f2749257657a1df7"
server_key_iv: "74ee0c2b7a81ee2f76016d917a79f440"
server_key_version: "encryptionkey@0"
I20260807 19:49:41.760928 23336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "8eee39f8ccf648cf9861460300596906"
format_stamp: "Formatted at 2026-08-07 19:49:41 on dist-test-slave-btw8"
server_key: "9ee4615e1e8f56c7f2749257657a1df7"
server_key_iv: "74ee0c2b7a81ee2f76016d917a79f440"
server_key_version: "encryptionkey@0"
I20260807 19:49:41.767773 23336 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:49:41.772681 23352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:41.774706 23336 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:49:41.775012 23336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "8eee39f8ccf648cf9861460300596906"
format_stamp: "Formatted at 2026-08-07 19:49:41 on dist-test-slave-btw8"
server_key: "9ee4615e1e8f56c7f2749257657a1df7"
server_key_iv: "74ee0c2b7a81ee2f76016d917a79f440"
server_key_version: "encryptionkey@0"
I20260807 19:49:41.775306 23336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:41.808841 23336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:41.814190 23336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:41.814553 23336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:41.836251 23336 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37545
I20260807 19:49:41.836283 23406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37545 every 8 connection(s)
I20260807 19:49:41.837858 23336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:41.842398 23407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:41.846053 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23336
I20260807 19:49:41.846297 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:41.846846 795 external_mini_cluster.cc:1428] Setting key b4ce4b7434a57cedd85eb87d4f5037dd
I20260807 19:49:41.856395 23407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906: Bootstrap starting.
I20260807 19:49:41.860234 23407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:41.861481 23407 log.cc:824] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:41.863559 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:41.849396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:58373 (local address 127.0.0.1:37545)
0807 19:49:41.850324 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:41.850338 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:41.850385 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:49:41.852529 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:41.852558 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:41.853088 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:41.853551 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:41.854224 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:41.855686 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:41.857440 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:41.858358 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:41.858646 (+ 288us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:41.860426 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:41.860439 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:41.860462 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:41.861100 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:41.861208 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:41.861245 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:41.861437 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:41.861776 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 19:49:41.861995 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":425,"threads_started":1}
I20260807 19:49:41.865489 23407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906: No bootstrap required, opened a new log
I20260807 19:49:41.868207 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:41.874918 23407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } }
I20260807 19:49:41.875506 23407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:41.875664 23407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eee39f8ccf648cf9861460300596906, State: Initialized, Role: FOLLOWER
I20260807 19:49:41.876456 23407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } }
I20260807 19:49:41.876834 23407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:41.877045 23407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:41.877238 23407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:41.880699 23407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } }
I20260807 19:49:41.881424 23407 leader_election.cc:302] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eee39f8ccf648cf9861460300596906; no voters:
I20260807 19:49:41.882391 23407 leader_election.cc:288] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:41.882452 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:41.873141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:54759 (local address 127.0.0.1:37545)
0807 19:49:41.873468 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:41.873479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:41.874057 (+ 578us) server_negotiation.cc:407] Connection header received
0807 19:49:41.874380 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:41.874388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:41.874486 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:41.874607 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:41.875409 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:41.876595 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:41.878543 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:41.879510 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:41.879680 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:41.881297 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:41.881302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:41.881306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:41.881370 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:41.881493 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:41.881520 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:41.881694 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:41.882047 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 19:49:41.882115 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:49:41.882522 23419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:41.883625 23419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [term 1 LEADER]: Becoming Leader. State: Replica: 8eee39f8ccf648cf9861460300596906, State: Running, Role: LEADER
I20260807 19:49:41.884546 23419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } }
I20260807 19:49:41.888103 23419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eee39f8ccf648cf9861460300596906. Latest consensus state: current_term: 1 leader_uuid: "8eee39f8ccf648cf9861460300596906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } } }
I20260807 19:49:41.888698 23419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:41.888943 23421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eee39f8ccf648cf9861460300596906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee39f8ccf648cf9861460300596906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37545 } } }
I20260807 19:49:41.889866 23421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:41.892612 23407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:41.892896 23423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:41.899659 23423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:41.921191 23423 catalog_manager.cc:1347] Generated new cluster ID: 1e53d4cae20a487a8bbdc2617f842f4b
I20260807 19:49:41.921393 23423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:41.943519 23423 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:41.945570 23423 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:41.953405 23423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eee39f8ccf648cf9861460300596906: Generated new TSK 0
I20260807 19:49:41.954439 23423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:41.961956 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23336
W20260807 19:49:41.963272 23402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.63:58373
W20260807 19:49:41.964219 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23336: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:41.964395 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23336
W20260807 19:49:41.988821 23321 connection.cc:535] client connection to 127.0.0.1:37545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37545): Connection reset by peer (error 104)
2026-08-07T19:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:42 dist-test-slave-btw8 krb5kdc[23442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:42 dist-test-slave-btw8 krb5kdc[23442](info): set up 2 sockets
Aug 07 19:49:42 dist-test-slave-btw8 krb5kdc[23442](info): commencing operation
krb5kdc: starting...
W20260807 19:49:43.217016 795 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:43 dist-test-slave-btw8 krb5kdc[23442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:43.616408 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:44.007016 23455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:44.007469 23455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:44.007587 23455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:44.015632 23455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:44.015775 23455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:44.015823 23455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:44.015895 23455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:44.015940 23455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:44.027704 23455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:44.029146 23455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:44.030494 23455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:44.045945 23460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:44.046226 23464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:44.046008 23461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:44.047416 23455 server_base.cc:1034] running on GCE node
I20260807 19:49:44.049111 23455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:44.051460 23455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:44.052769 23455 hybrid_clock.cc:648] HybridClock initialized: now 1786132184052721 us; error 49 us; skew 500 ppm
I20260807 19:49:44.053201 23455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:44 dist-test-slave-btw8 krb5kdc[23442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132184, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:44.061581 23455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:44.063757 23455 webserver.cc:460] Webserver started at http://127.0.0.1:35229/ using document root <none> and password file <none>
I20260807 19:49:44.064584 23455 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:44.064728 23455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:44.065052 23455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:44.068590 23455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f"
format_stamp: "Formatted at 2026-08-07 19:49:44 on dist-test-slave-btw8"
server_key: "f019327da53e4b37e584705ce6a2efbf"
server_key_iv: "67e72b018a3a47ff45ee8d09f055b3e3"
server_key_version: "encryptionkey@0"
I20260807 19:49:44.069610 23455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f"
format_stamp: "Formatted at 2026-08-07 19:49:44 on dist-test-slave-btw8"
server_key: "f019327da53e4b37e584705ce6a2efbf"
server_key_iv: "67e72b018a3a47ff45ee8d09f055b3e3"
server_key_version: "encryptionkey@0"
I20260807 19:49:44.076822 23455 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260807 19:49:44.082705 23471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:44.084646 23455 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:49:44.084905 23455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f"
format_stamp: "Formatted at 2026-08-07 19:49:44 on dist-test-slave-btw8"
server_key: "f019327da53e4b37e584705ce6a2efbf"
server_key_iv: "67e72b018a3a47ff45ee8d09f055b3e3"
server_key_version: "encryptionkey@0"
I20260807 19:49:44.085175 23455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:44.092432 23455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:44.097654 23455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:44.098017 23455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:44.121721 23455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42729
I20260807 19:49:44.121754 23523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42729 every 8 connection(s)
I20260807 19:49:44.123438 23455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:44.128064 23524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:44.131019 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23455
I20260807 19:49:44.131312 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:44.132165 795 external_mini_cluster.cc:1428] Setting key da3318578f14611dcfae5a76cc88c595
I20260807 19:49:44.143172 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:44.134846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:55541 (local address 127.0.0.1:42729)
0807 19:49:44.136053 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:44.136068 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:44.136124 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:49:44.138200 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:44.138232 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:44.138684 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:44.139373 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:44.141100 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:44.141111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:44.141137 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:44.141157 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:44.141239 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:44.141266 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:44.141571 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:44.141755 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 19:49:44.141996 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":599,"threads_started":1}
I20260807 19:49:44.143707 23524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f: Bootstrap starting.
I20260807 19:49:44.147996 23524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:44.148430 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:44.149422 23524 log.cc:824] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:44.153172 23524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f: No bootstrap required, opened a new log
I20260807 19:49:44.157300 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:44.154041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:38467 (local address 127.0.0.1:42729)
0807 19:49:44.154268 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:44.154276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:44.154737 (+ 461us) server_negotiation.cc:407] Connection header received
0807 19:49:44.154977 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:44.154982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:44.155036 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:44.155146 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:44.156390 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:44.156396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:44.156400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:44.156422 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:44.156480 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:44.156500 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:44.156679 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:44.156878 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 19:49:44.156964 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:49:44.163693 23524 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260807 19:49:44.164229 23524 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:44.164358 23524 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca31adf3dd064f3c9fd7d40f4f7d493f, State: Initialized, Role: FOLLOWER
I20260807 19:49:44.165158 23524 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260807 19:49:44.165539 23524 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:44.165732 23524 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:44.165939 23524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:44.168699 23524 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260807 19:49:44.169324 23524 leader_election.cc:302] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca31adf3dd064f3c9fd7d40f4f7d493f; no voters:
I20260807 19:49:44.170070 23524 leader_election.cc:288] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:44.170291 23535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:44.171181 23535 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [term 1 LEADER]: Becoming Leader. State: Replica: ca31adf3dd064f3c9fd7d40f4f7d493f, State: Running, Role: LEADER
I20260807 19:49:44.171857 23535 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260807 19:49:44.172667 23524 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:44.175669 23537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca31adf3dd064f3c9fd7d40f4f7d493f. Latest consensus state: current_term: 1 leader_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } } }
I20260807 19:49:44.175675 23536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca31adf3dd064f3c9fd7d40f4f7d493f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } } }
I20260807 19:49:44.176453 23537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:44.176448 23536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:44.181231 23541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:44.188452 23541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:44.200932 23541 catalog_manager.cc:1347] Generated new cluster ID: 26a80ecc9fef44d3a600dee0d5ad3f82
I20260807 19:49:44.201179 23541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:44.214339 23541 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:44.215013 23541 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:44.224653 23541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca31adf3dd064f3c9fd7d40f4f7d493f: Generated new TSK 0
I20260807 19:49:44.225713 23541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:44.244870 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23455
W20260807 19:49:44.246251 23519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.63:55541
W20260807 19:49:44.247221 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23455: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:44.247448 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23455
2026-08-07T19:49:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:44 dist-test-slave-btw8 krb5kdc[23558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:44 dist-test-slave-btw8 krb5kdc[23558](info): set up 2 sockets
Aug 07 19:49:44 dist-test-slave-btw8 krb5kdc[23558](info): commencing operation
krb5kdc: starting...
W20260807 19:49:45.488745 795 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.008s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:45 dist-test-slave-btw8 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:45.887527 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:46.281760 23574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:46.282181 23574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:46.282366 23574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:46.290288 23574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:46.290490 23574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:46.290621 23574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:46.290735 23574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:46.290830 23574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:46.301610 23574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:46.303102 23574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:46.304574 23574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:46.318142 23583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:46.318171 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:46.318105 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:46.318631 23574 server_base.cc:1034] running on GCE node
I20260807 19:49:46.319753 23574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:46.321434 23574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:46.322749 23574 hybrid_clock.cc:648] HybridClock initialized: now 1786132186322672 us; error 55 us; skew 500 ppm
I20260807 19:49:46.323208 23574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:46 dist-test-slave-btw8 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132186, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:46.327059 23574 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:46.329056 23574 webserver.cc:460] Webserver started at http://127.0.0.1:42533/ using document root <none> and password file <none>
I20260807 19:49:46.329876 23574 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:46.330047 23574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:46.330416 23574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:46.334091 23574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "7f19af45d2c347b9a31a296c2c8600dd"
format_stamp: "Formatted at 2026-08-07 19:49:46 on dist-test-slave-btw8"
server_key: "46b51ea674f0b0c8fb05eee10b1b16ed"
server_key_iv: "daca368dd6c4ffcd588831fcc80894b8"
server_key_version: "encryptionkey@0"
I20260807 19:49:46.335098 23574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "7f19af45d2c347b9a31a296c2c8600dd"
format_stamp: "Formatted at 2026-08-07 19:49:46 on dist-test-slave-btw8"
server_key: "46b51ea674f0b0c8fb05eee10b1b16ed"
server_key_iv: "daca368dd6c4ffcd588831fcc80894b8"
server_key_version: "encryptionkey@0"
I20260807 19:49:46.341770 23574 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:49:46.346215 23592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:46.348131 23574 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:46.348382 23574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "7f19af45d2c347b9a31a296c2c8600dd"
format_stamp: "Formatted at 2026-08-07 19:49:46 on dist-test-slave-btw8"
server_key: "46b51ea674f0b0c8fb05eee10b1b16ed"
server_key_iv: "daca368dd6c4ffcd588831fcc80894b8"
server_key_version: "encryptionkey@0"
I20260807 19:49:46.348632 23574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:46.368948 23574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:46.374095 23574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:46.374451 23574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:46.396893 23574 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38981
I20260807 19:49:46.396924 23643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38981 every 8 connection(s)
I20260807 19:49:46.398519 23574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:46.401726 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23574
I20260807 19:49:46.401932 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:46.402508 795 external_mini_cluster.cc:1428] Setting key 6c9f348c5eda9ae2d12fc4cb21313cc7
I20260807 19:49:46.403149 23644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:46.420013 23644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: Bootstrap starting.
Aug 07 19:49:46 dist-test-slave-btw8 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:49:46.424945 23644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:46.426765 23644 log.cc:824] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:46.431442 23644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: No bootstrap required, opened a new log
I20260807 19:49:46.433914 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:46.404773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:56253 (local address 127.0.0.1:38981)
0807 19:49:46.405861 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:46.405878 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:46.406159 (+ 281us) server_negotiation.cc:407] Connection header received
0807 19:49:46.408558 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:46.408583 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:46.409155 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:46.409528 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:46.411655 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:46.413080 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:46.415059 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:46.416109 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:46.416300 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:46.422158 (+ 5858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:46.422171 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:46.422195 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:46.422269 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:46.427269 (+ 5000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:46.428890 (+ 1621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:46.428901 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:46.428911 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:46.429204 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:46.429884 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:46.429890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:46.429894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:46.430795 (+ 901us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:46.431183 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:46.431796 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 19:49:46.432253 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":436,"threads_started":1}
I20260807 19:49:46.439313 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:46.442572 23644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260807 19:49:46.443141 23644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:46.443287 23644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f19af45d2c347b9a31a296c2c8600dd, State: Initialized, Role: FOLLOWER
I20260807 19:49:46.444092 23644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260807 19:49:46.444459 23644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:46.444696 23644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:46.444953 23644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:46.447745 23644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260807 19:49:46.448379 23644 leader_election.cc:302] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f19af45d2c347b9a31a296c2c8600dd; no voters:
I20260807 19:49:46.449209 23644 leader_election.cc:288] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:46.449334 23654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:46.452597 23644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:46.457404 23654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [term 1 LEADER]: Becoming Leader. State: Replica: 7f19af45d2c347b9a31a296c2c8600dd, State: Running, Role: LEADER
I20260807 19:49:46.458477 23654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260807 19:49:46.463425 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:46.444437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:50597 (local address 127.0.0.1:38981)
0807 19:49:46.444627 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:46.444656 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:46.445340 (+ 684us) server_negotiation.cc:407] Connection header received
0807 19:49:46.445583 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:46.445588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:46.445681 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:46.445798 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:46.447415 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:46.448271 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:46.449933 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:46.450637 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:46.450741 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:46.454199 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:46.454204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:46.454207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:46.454246 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:46.459896 (+ 5650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:46.460936 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:46.460943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:46.460949 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:46.461117 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:46.461723 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:46.461729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:46.461732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:46.462075 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:46.462391 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:46.462801 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 19:49:46.463014 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:49:46.470458 23656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f19af45d2c347b9a31a296c2c8600dd. Latest consensus state: current_term: 1 leader_uuid: "7f19af45d2c347b9a31a296c2c8600dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260807 19:49:46.470566 23655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f19af45d2c347b9a31a296c2c8600dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f19af45d2c347b9a31a296c2c8600dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260807 19:49:46.471006 23655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:46.471006 23656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd [sys.catalog]: This master's current role is: LEADER
W20260807 19:49:46.474267 23670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:49:46.474428 23670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:49:46.474594 23672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:46.477023 23672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:46.487670 23672 catalog_manager.cc:1347] Generated new cluster ID: ba2264292f8b4f9497fc625499104871
I20260807 19:49:46.487835 23672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:46.515194 23672 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:46.516665 23672 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:46.530205 23672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f19af45d2c347b9a31a296c2c8600dd: Generated new TSK 0
I20260807 19:49:46.531483 23672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:46.542003 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23574
I20260807 19:49:46.543787 23639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:46.642983 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23574
W20260807 19:49:46.665437 23556 connection.cc:535] client connection to 127.0.0.1:38981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38981): Connection reset by peer (error 104)
2026-08-07T19:49:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:46 dist-test-slave-btw8 krb5kdc[23678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:46 dist-test-slave-btw8 krb5kdc[23678](info): set up 2 sockets
Aug 07 19:49:46 dist-test-slave-btw8 krb5kdc[23678](info): commencing operation
krb5kdc: starting...
W20260807 19:49:47.873286 795 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:48 dist-test-slave-btw8 krb5kdc[23678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:48.272555 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:48.663749 23693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:48.664295 23693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:48.664526 23693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:48.672667 23693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:48.672813 23693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:48.672931 23693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:48.673043 23693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:48.673115 23693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:48.683990 23693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:48.685496 23693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:48.686990 23693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:48.700831 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:48.701021 23700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:48.701148 23699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:48.701948 23693 server_base.cc:1034] running on GCE node
I20260807 19:49:48.702913 23693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:48.704643 23693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:48.705905 23693 hybrid_clock.cc:648] HybridClock initialized: now 1786132188705868 us; error 39 us; skew 500 ppm
I20260807 19:49:48.706312 23693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:48 dist-test-slave-btw8 krb5kdc[23678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:48.709982 23693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:48.711899 23693 webserver.cc:460] Webserver started at http://127.0.0.1:45299/ using document root <none> and password file <none>
I20260807 19:49:48.712706 23693 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:48.712872 23693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:48.713207 23693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:48.716694 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "b673be896b9841a78f059c1611b1248b"
format_stamp: "Formatted at 2026-08-07 19:49:48 on dist-test-slave-btw8"
server_key: "ee1a1d589447caca7e4d30585ba3844e"
server_key_iv: "adbb3720798e406e285486285f4cd3ea"
server_key_version: "encryptionkey@0"
I20260807 19:49:48.717630 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "b673be896b9841a78f059c1611b1248b"
format_stamp: "Formatted at 2026-08-07 19:49:48 on dist-test-slave-btw8"
server_key: "ee1a1d589447caca7e4d30585ba3844e"
server_key_iv: "adbb3720798e406e285486285f4cd3ea"
server_key_version: "encryptionkey@0"
I20260807 19:49:48.723825 23693 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:49:48.728065 23711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:48.729740 23693 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:49:48.729979 23693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "b673be896b9841a78f059c1611b1248b"
format_stamp: "Formatted at 2026-08-07 19:49:48 on dist-test-slave-btw8"
server_key: "ee1a1d589447caca7e4d30585ba3844e"
server_key_iv: "adbb3720798e406e285486285f4cd3ea"
server_key_version: "encryptionkey@0"
I20260807 19:49:48.730229 23693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:48.745993 23693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:48.751085 23693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:48.751466 23693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:48.772677 23693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40907
I20260807 19:49:48.772804 23764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40907 every 8 connection(s)
I20260807 19:49:48.774225 23693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:48.778456 23765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:48.783275 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23693
I20260807 19:49:48.783569 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:48.784129 795 external_mini_cluster.cc:1428] Setting key c4303772be6de0e054671a727189ae64
I20260807 19:49:48.790802 23765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b: Bootstrap starting.
I20260807 19:49:48.795003 23765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:48.796433 23765 log.cc:824] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:48.800172 23765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b: No bootstrap required, opened a new log
I20260807 19:49:48.800518 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:48.786142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:42787 (local address 127.0.0.1:40907)
0807 19:49:48.787030 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:48.787043 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:48.787108 (+ 65us) server_negotiation.cc:407] Connection header received
0807 19:49:48.789226 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:48.789257 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:48.789740 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:48.790186 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:48.791134 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:48.792821 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:48.794920 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:48.795935 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:48.796113 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:48.797637 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:48.797651 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:48.797676 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:48.798421 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:48.798503 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:48.798536 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:48.798671 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:48.799056 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:49:48.799259 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":405,"threads_started":1}
I20260807 19:49:48.804828 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:48.810881 23765 raft_consensus.cc:348] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } }
I20260807 19:49:48.811537 23765 raft_consensus.cc:374] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:48.811681 23765 raft_consensus.cc:729] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b673be896b9841a78f059c1611b1248b, State: Initialized, Role: FOLLOWER
I20260807 19:49:48.812500 23765 consensus_queue.cc:260] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } }
I20260807 19:49:48.812801 23765 raft_consensus.cc:388] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:48.812960 23765 raft_consensus.cc:482] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:48.813234 23765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:48.817124 23765 raft_consensus.cc:504] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } }
I20260807 19:49:48.818131 23765 leader_election.cc:302] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b673be896b9841a78f059c1611b1248b; no voters:
I20260807 19:49:48.819005 23765 leader_election.cc:288] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:48.819201 23776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:48.822570 23776 raft_consensus.cc:686] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [term 1 LEADER]: Becoming Leader. State: Replica: b673be896b9841a78f059c1611b1248b, State: Running, Role: LEADER
I20260807 19:49:48.823105 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:48.811033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:59467 (local address 127.0.0.1:40907)
0807 19:49:48.811262 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:48.811271 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:48.811821 (+ 550us) server_negotiation.cc:407] Connection header received
0807 19:49:48.812069 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:48.812075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:48.812170 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:48.812315 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:48.813224 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:48.814491 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:48.816146 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:48.817141 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:48.817584 (+ 443us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:48.819419 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:48.819428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:48.819433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:48.819468 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:48.819551 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:48.819585 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:48.819722 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:48.820340 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 19:49:48.820408 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:49:48.823396 23776 consensus_queue.cc:237] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } }
I20260807 19:49:48.823606 23765 sys_catalog.cc:564] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:48.827647 23778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b673be896b9841a78f059c1611b1248b. Latest consensus state: current_term: 1 leader_uuid: "b673be896b9841a78f059c1611b1248b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } } }
I20260807 19:49:48.828213 23778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:48.828892 23777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b673be896b9841a78f059c1611b1248b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673be896b9841a78f059c1611b1248b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40907 } } }
I20260807 19:49:48.829411 23777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:48.831130 23785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:48.839394 23785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:48.853092 23785 catalog_manager.cc:1347] Generated new cluster ID: 094ebbeeedb74dbcb59378aa9bbe52c8
I20260807 19:49:48.853277 23785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:48.883385 23785 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:48.884876 23785 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:48.894676 23785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b673be896b9841a78f059c1611b1248b: Generated new TSK 0
I20260807 19:49:48.895645 23785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:48.911863 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23693
W20260807 19:49:48.913224 23760 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.63:42787
W20260807 19:49:48.914215 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23693: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:48.914418 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23693
W20260807 19:49:48.938331 23676 connection.cc:535] client connection to 127.0.0.1:40907 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40907): Connection reset by peer (error 104)
2026-08-07T19:49:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:49 dist-test-slave-btw8 krb5kdc[23799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:49 dist-test-slave-btw8 krb5kdc[23799](info): set up 2 sockets
Aug 07 19:49:49 dist-test-slave-btw8 krb5kdc[23799](info): commencing operation
krb5kdc: starting...
W20260807 19:49:50.187901 795 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.005s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:50 dist-test-slave-btw8 krb5kdc[23799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:49:50.589102 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:50.977085 23813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:50.977514 23813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:50.977725 23813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:50.985778 23813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:50.985937 23813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:50.986057 23813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:50.986181 23813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:50.986246 23813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:50.997493 23813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:50.999096 23813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:51.000747 23813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:51.014914 23820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:51.014878 23822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:51.014938 23819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:51.015810 23813 server_base.cc:1034] running on GCE node
I20260807 19:49:51.016891 23813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:51.018796 23813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:51.020125 23813 hybrid_clock.cc:648] HybridClock initialized: now 1786132191020066 us; error 47 us; skew 500 ppm
I20260807 19:49:51.020613 23813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:51 dist-test-slave-btw8 krb5kdc[23799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:51.024762 23813 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:49:51.026971 23813 webserver.cc:460] Webserver started at http://127.0.0.1:40569/ using document root <none> and password file <none>
I20260807 19:49:51.028628 23813 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:51.028962 23813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:51.029410 23813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:51.033651 23813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "3e6ebe67f569496d915fc30f19fc5ca2"
format_stamp: "Formatted at 2026-08-07 19:49:51 on dist-test-slave-btw8"
server_key: "3f0cc42888bd0a6a754147265944b3b1"
server_key_iv: "f94ec3b281d130c2c6865b4987c54a94"
server_key_version: "encryptionkey@0"
I20260807 19:49:51.034930 23813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "3e6ebe67f569496d915fc30f19fc5ca2"
format_stamp: "Formatted at 2026-08-07 19:49:51 on dist-test-slave-btw8"
server_key: "3f0cc42888bd0a6a754147265944b3b1"
server_key_iv: "f94ec3b281d130c2c6865b4987c54a94"
server_key_version: "encryptionkey@0"
I20260807 19:49:51.045042 23813 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.002s
I20260807 19:49:51.050666 23830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:51.052908 23813 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:49:51.053212 23813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "3e6ebe67f569496d915fc30f19fc5ca2"
format_stamp: "Formatted at 2026-08-07 19:49:51 on dist-test-slave-btw8"
server_key: "3f0cc42888bd0a6a754147265944b3b1"
server_key_iv: "f94ec3b281d130c2c6865b4987c54a94"
server_key_version: "encryptionkey@0"
I20260807 19:49:51.053503 23813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:51.061937 23813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:51.069499 23813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:51.069945 23813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:51.094156 23813 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40335
I20260807 19:49:51.094264 23886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40335 every 8 connection(s)
I20260807 19:49:51.095938 23813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:51.100298 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23813
I20260807 19:49:51.100454 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:51.100530 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:51.101135 795 external_mini_cluster.cc:1428] Setting key 1526ee02a29720405f6b6d0c736e999b
I20260807 19:49:51.110970 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:51.103573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:49037 (local address 127.0.0.1:40335)
0807 19:49:51.104587 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:51.104599 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:51.104655 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:49:51.107026 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:51.107059 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:51.107552 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:51.107964 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:51.109137 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:51.109144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:51.109157 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:51.109173 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:51.109239 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:51.109261 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:51.109471 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:51.109707 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 19:49:51.109979 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":457,"threads_started":1}
I20260807 19:49:51.115909 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:51.115902 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2: Bootstrap starting.
I20260807 19:49:51.120605 23887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:51.122155 23887 log.cc:824] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:51.125728 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:51.122074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.63:59417 (local address 127.0.0.1:40335)
0807 19:49:51.122327 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:51.122335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:51.122920 (+ 585us) server_negotiation.cc:407] Connection header received
0807 19:49:51.123161 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:51.123166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:51.123232 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:51.123329 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:51.124852 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:51.124861 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:51.124867 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:51.124889 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:51.124945 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:51.124967 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:51.125121 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:51.125311 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 19:49:51.125388 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:49:51.126467 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2: No bootstrap required, opened a new log
I20260807 19:49:51.135705 23887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } }
I20260807 19:49:51.136335 23887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:51.136482 23887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6ebe67f569496d915fc30f19fc5ca2, State: Initialized, Role: FOLLOWER
I20260807 19:49:51.137377 23887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } }
I20260807 19:49:51.137655 23887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:51.137830 23887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:51.137974 23887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:51.141856 23887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } }
I20260807 19:49:51.142684 23887 leader_election.cc:302] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e6ebe67f569496d915fc30f19fc5ca2; no voters:
I20260807 19:49:51.143659 23887 leader_election.cc:288] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:51.144064 23900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:51.145190 23900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [term 1 LEADER]: Becoming Leader. State: Replica: 3e6ebe67f569496d915fc30f19fc5ca2, State: Running, Role: LEADER
I20260807 19:49:51.145848 23900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } }
I20260807 19:49:51.146308 23887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:51.150274 23901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } } }
I20260807 19:49:51.152146 23901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:51.149684 23902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e6ebe67f569496d915fc30f19fc5ca2. Latest consensus state: current_term: 1 leader_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6ebe67f569496d915fc30f19fc5ca2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40335 } } }
I20260807 19:49:51.152979 23902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:51.155059 23909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:51.161203 23909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:51.172868 23909 catalog_manager.cc:1347] Generated new cluster ID: 9169b80fc579452b93d1aef35136910f
I20260807 19:49:51.173010 23909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:51.189968 23909 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:51.190598 23909 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:51.203320 23909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e6ebe67f569496d915fc30f19fc5ca2: Generated new TSK 0
I20260807 19:49:51.204742 23909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:51.220782 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23813
W20260807 19:49:51.222025 23881 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.63:49037
W20260807 19:49:51.222848 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23813: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:51.223044 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23813
2026-08-07T19:49:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:51 dist-test-slave-btw8 krb5kdc[23921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:51 dist-test-slave-btw8 krb5kdc[23921](info): set up 2 sockets
Aug 07 19:49:51 dist-test-slave-btw8 krb5kdc[23921](info): commencing operation
krb5kdc: starting...
W20260807 19:49:52.522805 795 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:52 dist-test-slave-btw8 krb5kdc[23921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:49:52.929339 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37241
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:37241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:53.335063 23936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:53.335587 23936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:53.335742 23936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:53.344966 23936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:53.345136 23936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:53.345191 23936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:53.345256 23936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:53.345328 23936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:53.358155 23936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:37241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:53.359779 23936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:53.361302 23936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:53.375766 23941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:53.375741 23944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:53.375953 23942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:53.377280 23936 server_base.cc:1034] running on GCE node
I20260807 19:49:53.378191 23936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:53.379949 23936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:53.381224 23936 hybrid_clock.cc:648] HybridClock initialized: now 1786132193381148 us; error 66 us; skew 500 ppm
I20260807 19:49:53.381632 23936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:53 dist-test-slave-btw8 krb5kdc[23921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:53.386359 23936 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:53.388608 23936 webserver.cc:460] Webserver started at http://127.0.198.254:38487/ using document root <none> and password file <none>
I20260807 19:49:53.389489 23936 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:53.389619 23936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:53.389978 23936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:53.393689 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "67d1655547ae4e108dcdabbd54b3cf7a"
format_stamp: "Formatted at 2026-08-07 19:49:53 on dist-test-slave-btw8"
server_key: "1f6a1490622b3ed259c0a9da8abdfb60"
server_key_iv: "0a35bb0d0ce3270e5f199e344c105e00"
server_key_version: "encryptionkey@0"
I20260807 19:49:53.394773 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "67d1655547ae4e108dcdabbd54b3cf7a"
format_stamp: "Formatted at 2026-08-07 19:49:53 on dist-test-slave-btw8"
server_key: "1f6a1490622b3ed259c0a9da8abdfb60"
server_key_iv: "0a35bb0d0ce3270e5f199e344c105e00"
server_key_version: "encryptionkey@0"
I20260807 19:49:53.401746 23936 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:49:53.407004 23952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:53.409704 23936 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:49:53.410027 23936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "67d1655547ae4e108dcdabbd54b3cf7a"
format_stamp: "Formatted at 2026-08-07 19:49:53 on dist-test-slave-btw8"
server_key: "1f6a1490622b3ed259c0a9da8abdfb60"
server_key_iv: "0a35bb0d0ce3270e5f199e344c105e00"
server_key_version: "encryptionkey@0"
I20260807 19:49:53.410285 23936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:53.469846 23936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:53.475137 23936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:53.475502 23936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:53.498489 23936 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:37241
I20260807 19:49:53.498545 24004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:37241 every 8 connection(s)
I20260807 19:49:53.500380 23936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:53.503885 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 23936
I20260807 19:49:53.504132 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:53.504676 795 external_mini_cluster.cc:1428] Setting key 35403eba480114f873ea83f0a097d14a
I20260807 19:49:53.505405 24005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:49:53 dist-test-slave-btw8 krb5kdc[23921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:49:53.522761 24005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a: Bootstrap starting.
I20260807 19:49:53.527061 24005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:53.528755 24005 log.cc:824] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:53.532702 24005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a: No bootstrap required, opened a new log
I20260807 19:49:53.534371 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:53.506935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46563 (local address 127.0.198.254:37241)
0807 19:49:53.507938 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:53.507952 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:53.508009 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:49:53.509690 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:53.509718 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:53.510220 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:53.510623 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:53.512955 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:53.515738 (+ 2783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:53.517680 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:53.518541 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:53.518842 (+ 301us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:53.524093 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:53.524103 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:53.524121 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:53.524175 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:53.528091 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:53.529701 (+ 1610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:53.529791 (+ 90us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:53.529807 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:53.530069 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:53.530752 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:53.530756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:53.530759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:53.531479 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:53.531880 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:53.532447 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 19:49:53.532959 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":521,"threads_started":1}
I20260807 19:49:53.539885 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:53.542523 24005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } }
I20260807 19:49:53.543048 24005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:53.543195 24005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d1655547ae4e108dcdabbd54b3cf7a, State: Initialized, Role: FOLLOWER
I20260807 19:49:53.544121 24005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } }
I20260807 19:49:53.544430 24005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:53.544602 24005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:53.544826 24005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:53.549036 24005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } }
I20260807 19:49:53.549808 24005 leader_election.cc:302] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d1655547ae4e108dcdabbd54b3cf7a; no voters:
I20260807 19:49:53.550613 24005 leader_election.cc:288] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:53.550704 24017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:53.553185 24005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:53.557618 24017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [term 1 LEADER]: Becoming Leader. State: Replica: 67d1655547ae4e108dcdabbd54b3cf7a, State: Running, Role: LEADER
I20260807 19:49:53.558439 24017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } }
I20260807 19:49:53.565627 24019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d1655547ae4e108dcdabbd54b3cf7a. Latest consensus state: current_term: 1 leader_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } } }
I20260807 19:49:53.566220 24019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:53.569051 24018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d1655547ae4e108dcdabbd54b3cf7a" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 37241 } } }
I20260807 19:49:53.569500 24018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:53.573659 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:53.545390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40169 (local address 127.0.198.254:37241)
0807 19:49:53.545667 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:53.545678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:53.546245 (+ 567us) server_negotiation.cc:407] Connection header received
0807 19:49:53.546508 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:53.546513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:53.546594 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:53.546689 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:53.549078 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:53.550117 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:53.552319 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:53.553455 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:53.553656 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:53.558406 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:53.558422 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:53.558431 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:49:53.558489 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:53.565157 (+ 6668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:53.566759 (+ 1602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:53.566768 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:53.566776 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:53.567050 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:49:53.568007 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:49:53.568016 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:49:53.568023 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:49:53.568577 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:53.572374 (+ 3797us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:53.572701 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:49:53.573028 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:49:53.577549 24028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:53.587816 24028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:53.600392 24028 catalog_manager.cc:1347] Generated new cluster ID: 449b19d71b2a4f839335839ca0ab3f6f
I20260807 19:49:53.600569 24028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:53.626089 24028 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:53.628657 24028 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:53.640923 24028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67d1655547ae4e108dcdabbd54b3cf7a: Generated new TSK 0
I20260807 19:49:53.642171 24028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:53.656441 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 23936
I20260807 19:49:53.658880 24000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:49:53.764164 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 23936
W20260807 19:49:53.786494 23919 connection.cc:535] client connection to 127.0.198.254:37241 recv error: Network error: failed to read from TLS socket (remote: 127.0.198.254:37241): Connection reset by peer (error 104)
2026-08-07T19:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:53 dist-test-slave-btw8 krb5kdc[24041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:53 dist-test-slave-btw8 krb5kdc[24041](info): set up 2 sockets
Aug 07 19:49:53 dist-test-slave-btw8 krb5kdc[24041](info): commencing operation
krb5kdc: starting...
W20260807 19:49:55.093861 795 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:55 dist-test-slave-btw8 krb5kdc[24041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:49:55.499929 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38409
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:38409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:55.888355 24056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:55.888738 24056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:55.888850 24056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:55.897600 24056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:55.897756 24056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:55.897805 24056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:55.897863 24056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:55.897909 24056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:55.913956 24056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:38409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:55.915809 24056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:55.917881 24056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:55.935014 24062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:55.935070 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:55.935873 24061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:55.936308 24056 server_base.cc:1034] running on GCE node
I20260807 19:49:55.937678 24056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:55.939707 24056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:55.941004 24056 hybrid_clock.cc:648] HybridClock initialized: now 1786132195940969 us; error 39 us; skew 500 ppm
I20260807 19:49:55.941393 24056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:55 dist-test-slave-btw8 krb5kdc[24041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:55.945019 24056 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:55.947093 24056 webserver.cc:460] Webserver started at http://127.0.198.254:43601/ using document root <none> and password file <none>
I20260807 19:49:55.948056 24056 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:55.948253 24056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:55.948645 24056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:55.952353 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "3dff3b630d7e4c3cb7e1877f96066e31"
format_stamp: "Formatted at 2026-08-07 19:49:55 on dist-test-slave-btw8"
server_key: "d20aef006638e8930360e1296f4884f9"
server_key_iv: "ec02a07c67488072861058e7c066cef0"
server_key_version: "encryptionkey@0"
I20260807 19:49:55.953281 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "3dff3b630d7e4c3cb7e1877f96066e31"
format_stamp: "Formatted at 2026-08-07 19:49:55 on dist-test-slave-btw8"
server_key: "d20aef006638e8930360e1296f4884f9"
server_key_iv: "ec02a07c67488072861058e7c066cef0"
server_key_version: "encryptionkey@0"
I20260807 19:49:55.959317 24056 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:49:55.963313 24072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:55.964910 24056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:49:55.965118 24056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "3dff3b630d7e4c3cb7e1877f96066e31"
format_stamp: "Formatted at 2026-08-07 19:49:55 on dist-test-slave-btw8"
server_key: "d20aef006638e8930360e1296f4884f9"
server_key_iv: "ec02a07c67488072861058e7c066cef0"
server_key_version: "encryptionkey@0"
I20260807 19:49:55.965373 24056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:55.990357 24056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:55.995607 24056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:55.995976 24056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:56.018949 24056 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:38409
I20260807 19:49:56.018975 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:38409 every 8 connection(s)
I20260807 19:49:56.020586 24056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:56.022634 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24056
I20260807 19:49:56.022863 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:56.023566 795 external_mini_cluster.cc:1428] Setting key f820c52a4c12c2b9294acb034562aed3
I20260807 19:49:56.026018 24129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:56.039053 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:56.025713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42711 (local address 127.0.198.254:38409)
0807 19:49:56.026729 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:56.026742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:56.026790 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:49:56.028729 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:56.028757 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:56.029231 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:56.029740 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:56.030426 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:56.032069 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:56.033650 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:56.034474 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:56.034706 (+ 232us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:56.036334 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:56.036342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:56.036360 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:56.037024 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:56.037123 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:56.037159 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:56.037402 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:56.037756 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 19:49:56.037954 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":520,"threads_started":1}
I20260807 19:49:56.039935 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31: Bootstrap starting.
I20260807 19:49:56.043428 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:56.043713 24129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:56.044838 24129 log.cc:824] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:56.048144 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31: No bootstrap required, opened a new log
I20260807 19:49:56.056207 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:56.048437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56459 (local address 127.0.198.254:38409)
0807 19:49:56.048673 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:56.048680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:56.049107 (+ 427us) server_negotiation.cc:407] Connection header received
0807 19:49:56.049321 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:56.049325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:56.049387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:56.049472 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:56.050134 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:56.050971 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:56.053018 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:49:56.053546 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:49:56.053690 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:49:56.055085 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:56.055089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:56.055093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:56.055124 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:56.055168 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:56.055212 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:56.055303 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:56.055610 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:49:56.055809 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:49:56.058920 24129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } }
I20260807 19:49:56.059424 24129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:56.059549 24129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dff3b630d7e4c3cb7e1877f96066e31, State: Initialized, Role: FOLLOWER
I20260807 19:49:56.060256 24129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } }
I20260807 19:49:56.060520 24129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:56.060684 24129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:56.060877 24129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:56.063644 24129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } }
I20260807 19:49:56.064395 24129 leader_election.cc:302] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dff3b630d7e4c3cb7e1877f96066e31; no voters:
I20260807 19:49:56.065181 24129 leader_election.cc:288] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:56.065340 24139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:56.066093 24139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [term 1 LEADER]: Becoming Leader. State: Replica: 3dff3b630d7e4c3cb7e1877f96066e31, State: Running, Role: LEADER
I20260807 19:49:56.066635 24139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } }
I20260807 19:49:56.067548 24129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:56.069876 24141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dff3b630d7e4c3cb7e1877f96066e31. Latest consensus state: current_term: 1 leader_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } } }
I20260807 19:49:56.070556 24141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:56.069976 24140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dff3b630d7e4c3cb7e1877f96066e31" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38409 } } }
I20260807 19:49:56.072659 24140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:56.077092 24145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:56.083844 24145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:56.094920 24145 catalog_manager.cc:1347] Generated new cluster ID: 186aa87d61fc463a95fb14923e19b130
I20260807 19:49:56.095093 24145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:56.124123 24145 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:56.125711 24145 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:56.142764 24145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dff3b630d7e4c3cb7e1877f96066e31: Generated new TSK 0
I20260807 19:49:56.143888 24145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:56.156883 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24056
W20260807 19:49:56.158277 24124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42711
W20260807 19:49:56.159169 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24056: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:56.159366 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24056
W20260807 19:49:56.181392 24038 connection.cc:535] client connection to 127.0.198.254:38409 recv error: Network error: failed to read from TLS socket (remote: 127.0.198.254:38409): Connection reset by peer (error 104)
2026-08-07T19:49:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:56 dist-test-slave-btw8 krb5kdc[24160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:56 dist-test-slave-btw8 krb5kdc[24160](info): set up 2 sockets
Aug 07 19:49:56 dist-test-slave-btw8 krb5kdc[24160](info): commencing operation
krb5kdc: starting...
W20260807 19:49:57.472966 795 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.003s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:49:57 dist-test-slave-btw8 krb5kdc[24160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:49:57.863276 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33013
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:33013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:49:58.256716 24173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:49:58.257066 24173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:49:58.257220 24173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:49:58.265370 24173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:49:58.265517 24173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:49:58.265599 24173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:49:58.265727 24173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:49:58.265795 24173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:49:58.276943 24173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:33013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:33013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:49:58.278417 24173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:49:58.279856 24173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:49:58.293996 24179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:58.294100 24180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:49:58.294327 24182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:49:58.295279 24173 server_base.cc:1034] running on GCE node
I20260807 19:49:58.296320 24173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:49:58.297982 24173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:49:58.299284 24173 hybrid_clock.cc:648] HybridClock initialized: now 1786132198299215 us; error 74 us; skew 500 ppm
I20260807 19:49:58.299732 24173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:49:58 dist-test-slave-btw8 krb5kdc[24160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:49:58.303604 24173 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:49:58.305487 24173 webserver.cc:460] Webserver started at http://127.0.198.254:43751/ using document root <none> and password file <none>
I20260807 19:49:58.306348 24173 fs_manager.cc:359] Metadata directory not provided
I20260807 19:49:58.306525 24173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:49:58.306890 24173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:49:58.310566 24173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "aa8f8d1e29d3411d81697ff91d533831"
format_stamp: "Formatted at 2026-08-07 19:49:58 on dist-test-slave-btw8"
server_key: "fe70fb0a0b290a13c7c4699d6ec66e8d"
server_key_iv: "e86e8ce6b31d97b78ff12ed74fda206c"
server_key_version: "encryptionkey@0"
I20260807 19:49:58.311755 24173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "aa8f8d1e29d3411d81697ff91d533831"
format_stamp: "Formatted at 2026-08-07 19:49:58 on dist-test-slave-btw8"
server_key: "fe70fb0a0b290a13c7c4699d6ec66e8d"
server_key_iv: "e86e8ce6b31d97b78ff12ed74fda206c"
server_key_version: "encryptionkey@0"
I20260807 19:49:58.318617 24173 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:49:58.323798 24189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:49:58.325798 24173 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:49:58.326200 24173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "aa8f8d1e29d3411d81697ff91d533831"
format_stamp: "Formatted at 2026-08-07 19:49:58 on dist-test-slave-btw8"
server_key: "fe70fb0a0b290a13c7c4699d6ec66e8d"
server_key_iv: "e86e8ce6b31d97b78ff12ed74fda206c"
server_key_version: "encryptionkey@0"
I20260807 19:49:58.326485 24173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:49:58.334169 24173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:49:58.340552 24173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:49:58.340970 24173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:49:58.365075 24173 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:33013
I20260807 19:49:58.365134 24246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:33013 every 8 connection(s)
I20260807 19:49:58.366941 24173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:49:58.371819 24247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:58.375099 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24173
I20260807 19:49:58.375320 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:49:58.375922 795 external_mini_cluster.cc:1428] Setting key d45ad12021032039edee43b744ec44a7
I20260807 19:49:58.385314 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:58.378166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54549 (local address 127.0.198.254:33013)
0807 19:49:58.379138 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:58.379152 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:58.379199 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:49:58.381143 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:58.381170 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:58.381562 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:58.382052 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:58.383265 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:58.383272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:58.383289 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:58.383305 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:58.383454 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:58.383480 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:58.383814 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:58.383980 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 19:49:58.384217 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":451,"threads_started":1}
I20260807 19:49:58.385943 24247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831: Bootstrap starting.
I20260807 19:49:58.389526 24247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831: Neither blocks nor log segments found. Creating new log.
I20260807 19:49:58.389792 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:49:58.390928 24247 log.cc:824] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831: Log is configured to *not* fsync() on all Append() calls
I20260807 19:49:58.394829 24247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831: No bootstrap required, opened a new log
I20260807 19:49:58.399461 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:49:58.395402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43021 (local address 127.0.198.254:33013)
0807 19:49:58.395682 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 19:49:58.395690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:49:58.396230 (+ 540us) server_negotiation.cc:407] Connection header received
0807 19:49:58.396427 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:49:58.396431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:49:58.396487 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:49:58.396587 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:49:58.398538 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:49:58.398543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:49:58.398547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:49:58.398571 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:49:58.398630 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:49:58.398649 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:49:58.398828 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:49:58.398940 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:49:58.399003 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:49:58.404803 24247 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } }
I20260807 19:49:58.405310 24247 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:49:58.405432 24247 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8f8d1e29d3411d81697ff91d533831, State: Initialized, Role: FOLLOWER
I20260807 19:49:58.406291 24247 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } }
I20260807 19:49:58.406594 24247 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:49:58.406764 24247 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:49:58.406987 24247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:49:58.409621 24247 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } }
I20260807 19:49:58.410341 24247 leader_election.cc:302] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa8f8d1e29d3411d81697ff91d533831; no voters:
I20260807 19:49:58.411114 24247 leader_election.cc:288] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:49:58.411250 24257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:49:58.412011 24257 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [term 1 LEADER]: Becoming Leader. State: Replica: aa8f8d1e29d3411d81697ff91d533831, State: Running, Role: LEADER
I20260807 19:49:58.412706 24257 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } }
I20260807 19:49:58.413741 24247 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:49:58.416045 24258 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa8f8d1e29d3411d81697ff91d533831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } } }
I20260807 19:49:58.416764 24258 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:58.416646 24259 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa8f8d1e29d3411d81697ff91d533831. Latest consensus state: current_term: 1 leader_uuid: "aa8f8d1e29d3411d81697ff91d533831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8f8d1e29d3411d81697ff91d533831" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 33013 } } }
I20260807 19:49:58.417287 24259 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:49:58.423559 24265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:49:58.428764 24265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:49:58.439851 24265 catalog_manager.cc:1347] Generated new cluster ID: 3e9b7721c8c84bea9cd2d6cb9b7b8166
I20260807 19:49:58.440016 24265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:49:58.472317 24265 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:49:58.472864 24265 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:49:58.487609 24265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa8f8d1e29d3411d81697ff91d533831: Generated new TSK 0
I20260807 19:49:58.488690 24265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:49:58.494105 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24173
W20260807 19:49:58.496321 24242 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54549
W20260807 19:49:58.497411 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24173: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:49:58.497624 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24173
2026-08-07T19:49:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:49:58 dist-test-slave-btw8 krb5kdc[24279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:49:58 dist-test-slave-btw8 krb5kdc[24279](info): set up 2 sockets
Aug 07 19:49:58 dist-test-slave-btw8 krb5kdc[24279](info): commencing operation
krb5kdc: starting...
W20260807 19:49:59.742228 795 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.002s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:50:00 dist-test-slave-btw8 krb5kdc[24279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:50:00.172672 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38617
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:33481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:38617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:50:00.583815 24292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:00.584157 24292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:00.584264 24292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:00.592959 24292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:00.593142 24292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:00.593197 24292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:00.593273 24292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:00.593330 24292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:00.605172 24292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:33481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:38617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:38617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:00.606626 24292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:00.608194 24292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:00.623044 24299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:00.622980 24301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:00.623456 24298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:00.624245 24292 server_base.cc:1034] running on GCE node
I20260807 19:50:00.625249 24292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:00.626995 24292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:00.628259 24292 hybrid_clock.cc:648] HybridClock initialized: now 1786132200628179 us; error 82 us; skew 500 ppm
I20260807 19:50:00.628742 24292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:00 dist-test-slave-btw8 krb5kdc[24279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:00.632840 24292 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:50:00.634814 24292 webserver.cc:460] Webserver started at http://127.0.198.254:33463/ using document root <none> and password file <none>
I20260807 19:50:00.635756 24292 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:00.635883 24292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:00.636178 24292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:00.640709 24292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "a300d49e473445279f1c27502157c188"
format_stamp: "Formatted at 2026-08-07 19:50:00 on dist-test-slave-btw8"
server_key: "751838b92fd51fef1625aaa12681f77d"
server_key_iv: "7b2b0ca5be569b57e1e5f11da96a5d59"
server_key_version: "encryptionkey@0"
I20260807 19:50:00.641760 24292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "a300d49e473445279f1c27502157c188"
format_stamp: "Formatted at 2026-08-07 19:50:00 on dist-test-slave-btw8"
server_key: "751838b92fd51fef1625aaa12681f77d"
server_key_iv: "7b2b0ca5be569b57e1e5f11da96a5d59"
server_key_version: "encryptionkey@0"
I20260807 19:50:00.648818 24292 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 19:50:00.653882 24309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:00.655936 24292 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:50:00.656209 24292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "a300d49e473445279f1c27502157c188"
format_stamp: "Formatted at 2026-08-07 19:50:00 on dist-test-slave-btw8"
server_key: "751838b92fd51fef1625aaa12681f77d"
server_key_iv: "7b2b0ca5be569b57e1e5f11da96a5d59"
server_key_version: "encryptionkey@0"
I20260807 19:50:00.656461 24292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:00.676824 24292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:00.682343 24292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:00.682736 24292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:00.704241 24292 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:38617
I20260807 19:50:00.704242 24361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:38617 every 8 connection(s)
I20260807 19:50:00.705845 24292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:50:00.710546 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:00.715974 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24292
I20260807 19:50:00.716221 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:50:00.716818 795 external_mini_cluster.cc:1428] Setting key 5f32129305ff35c53c0f808b0cabdd57
I20260807 19:50:00.724220 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188: Bootstrap starting.
I20260807 19:50:00.727959 24362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:00.729318 24362 log.cc:824] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:00.733372 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188: No bootstrap required, opened a new log
Aug 07 19:50:00 dist-test-slave-btw8 krb5kdc[24279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:00.744640 24362 raft_consensus.cc:348] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } }
I20260807 19:50:00.745132 24362 raft_consensus.cc:374] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:50:00.745311 24362 raft_consensus.cc:729] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a300d49e473445279f1c27502157c188, State: Initialized, Role: FOLLOWER
I20260807 19:50:00.746779 24362 consensus_queue.cc:260] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } }
I20260807 19:50:00.747174 24362 raft_consensus.cc:388] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:50:00.747390 24362 raft_consensus.cc:482] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:50:00.747613 24362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:00.750502 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:00.719241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41769 (local address 127.0.198.254:38617)
0807 19:50:00.720684 (+ 1443us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:00.720709 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:00.720785 (+ 76us) server_negotiation.cc:407] Connection header received
0807 19:50:00.723216 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:00.723253 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:00.723950 (+ 697us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:00.724467 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:00.726219 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:00.728339 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:00.730343 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:00.731521 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:00.731765 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:00.737958 (+ 6193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:00.737973 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:00.738005 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:00.738076 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:00.743094 (+ 5018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:00.744313 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:00.744323 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:00.744333 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:00.744608 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:00.745249 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:00.745255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:00.745260 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:00.746849 (+ 1589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:00.747226 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:00.748104 (+ 878us) server_negotiation.cc:299] Negotiation successful
0807 19:50:00.748705 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":609,"threads_started":1}
I20260807 19:50:00.751647 24362 raft_consensus.cc:504] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } }
I20260807 19:50:00.752317 24362 leader_election.cc:302] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a300d49e473445279f1c27502157c188; no voters:
I20260807 19:50:00.752951 24362 leader_election.cc:288] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:50:00.753191 24367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:50:00.754864 24367 raft_consensus.cc:686] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [term 1 LEADER]: Becoming Leader. State: Replica: a300d49e473445279f1c27502157c188, State: Running, Role: LEADER
I20260807 19:50:00.755415 24362 sys_catalog.cc:564] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:00.755714 24367 consensus_queue.cc:237] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } }
I20260807 19:50:00.756366 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:50:00.760004 24368 sys_catalog.cc:455] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a300d49e473445279f1c27502157c188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } } }
I20260807 19:50:00.760516 24368 sys_catalog.cc:458] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:00.760756 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a300d49e473445279f1c27502157c188. Latest consensus state: current_term: 1 leader_uuid: "a300d49e473445279f1c27502157c188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a300d49e473445279f1c27502157c188" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 38617 } } }
I20260807 19:50:00.761094 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:00.765641 24379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:50:00.776343 24379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:50:00.789697 24379 catalog_manager.cc:1347] Generated new cluster ID: fd6f8c30ac1a4c318a40eca29e1d3c43
I20260807 19:50:00.789916 24379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:50:00.797479 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:00.774640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40435 (local address 127.0.198.254:38617)
0807 19:50:00.774932 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:00.774942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:00.775747 (+ 805us) server_negotiation.cc:407] Connection header received
0807 19:50:00.775988 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:00.775994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:00.776087 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:00.776227 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:00.779154 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:00.780859 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:00.783660 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:00.784728 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:00.784875 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:00.790061 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:00.790072 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:00.790081 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:00.790147 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:00.794227 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:00.795111 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:00.795117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:00.795120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:00.795283 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:00.795770 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:00.795774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:00.795776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:00.796133 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:00.796419 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:00.796818 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 19:50:00.797082 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 19:50:00.805918 24379 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:50:00.807578 24379 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:50:00.819442 24379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a300d49e473445279f1c27502157c188: Generated new TSK 0
I20260807 19:50:00.820461 24379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:50:00.828574 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24292
I20260807 19:50:00.830513 24357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:00.943858 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24292
2026-08-07T19:50:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:50:01 dist-test-slave-btw8 krb5kdc[24396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:50:01 dist-test-slave-btw8 krb5kdc[24396](info): set up 2 sockets
Aug 07 19:50:01 dist-test-slave-btw8 krb5kdc[24396](info): commencing operation
krb5kdc: starting...
W20260807 19:50:02.241206 795 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:50:02 dist-test-slave-btw8 krb5kdc[24396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:50:02.653781 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36637
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:36637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:50:03.079847 24409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:03.080267 24409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:03.080377 24409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:03.090255 24409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:03.090421 24409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:03.090472 24409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:03.090540 24409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:03.090595 24409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:03.101614 24409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:36637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:36637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:03.103032 24409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:03.104697 24409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:03.119648 24415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:03.119712 24418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:03.119606 24416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:03.121917 24409 server_base.cc:1034] running on GCE node
I20260807 19:50:03.123126 24409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:03.125087 24409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:03.126437 24409 hybrid_clock.cc:648] HybridClock initialized: now 1786132203126363 us; error 77 us; skew 500 ppm
I20260807 19:50:03.127022 24409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:03 dist-test-slave-btw8 krb5kdc[24396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:03.131628 24409 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:50:03.134420 24409 webserver.cc:460] Webserver started at http://127.0.198.254:43131/ using document root <none> and password file <none>
I20260807 19:50:03.135425 24409 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:03.135632 24409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:03.136122 24409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:03.139860 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "c1ca289b211d458f9a35e6597ec281c1"
format_stamp: "Formatted at 2026-08-07 19:50:03 on dist-test-slave-btw8"
server_key: "30853a45f1f94e37081a742639e08d61"
server_key_iv: "ef5e1b77739f4d0a87d331e885fd4534"
server_key_version: "encryptionkey@0"
I20260807 19:50:03.140851 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "c1ca289b211d458f9a35e6597ec281c1"
format_stamp: "Formatted at 2026-08-07 19:50:03 on dist-test-slave-btw8"
server_key: "30853a45f1f94e37081a742639e08d61"
server_key_iv: "ef5e1b77739f4d0a87d331e885fd4534"
server_key_version: "encryptionkey@0"
I20260807 19:50:03.148221 24409 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 19:50:03.152962 24426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:03.154959 24409 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:50:03.155260 24409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "c1ca289b211d458f9a35e6597ec281c1"
format_stamp: "Formatted at 2026-08-07 19:50:03 on dist-test-slave-btw8"
server_key: "30853a45f1f94e37081a742639e08d61"
server_key_iv: "ef5e1b77739f4d0a87d331e885fd4534"
server_key_version: "encryptionkey@0"
I20260807 19:50:03.155614 24409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:03.189230 24409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:03.195454 24409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:03.195864 24409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:03.222637 24409 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:36637
I20260807 19:50:03.222652 24482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:36637 every 8 connection(s)
I20260807 19:50:03.224393 24409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:50:03.228948 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24409
I20260807 19:50:03.229172 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:03.229193 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:50:03.229796 795 external_mini_cluster.cc:1428] Setting key 1aaf106fdbd3641d22305e0c13caa74b
I20260807 19:50:03.245636 24483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1: Bootstrap starting.
I20260807 19:50:03.246263 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:03.232188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47691 (local address 127.0.198.254:36637)
0807 19:50:03.233212 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:03.233227 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:03.233280 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:50:03.235244 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:03.235273 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:03.235874 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:03.236302 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:03.237167 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:03.238780 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:03.240939 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:03.241813 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:03.242000 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:03.243588 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:03.243597 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:03.243615 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:50:03.244238 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:03.244331 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:50:03.244364 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:03.244468 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:03.244739 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 19:50:03.245056 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":496,"threads_started":1}
I20260807 19:50:03.249193 24483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:03.250406 24483 log.cc:824] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:03.250485 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:50:03.254158 24483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1: No bootstrap required, opened a new log
I20260807 19:50:03.264894 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:03.256262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58577 (local address 127.0.198.254:36637)
0807 19:50:03.256477 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:03.256488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:03.256935 (+ 447us) server_negotiation.cc:407] Connection header received
0807 19:50:03.257089 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:03.257094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:03.257195 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:03.257312 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:03.257838 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:03.258977 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:03.260476 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:03.261262 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:03.261401 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:03.263017 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:03.263061 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:03.263068 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:50:03.263101 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:03.263235 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:50:03.263272 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:03.263462 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:03.263629 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 19:50:03.263862 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:50:03.264977 24483 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } }
I20260807 19:50:03.265962 24483 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:50:03.266157 24483 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ca289b211d458f9a35e6597ec281c1, State: Initialized, Role: FOLLOWER
I20260807 19:50:03.267295 24483 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } }
I20260807 19:50:03.267683 24483 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:50:03.267850 24483 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:50:03.268090 24483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:03.270673 24483 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } }
I20260807 19:50:03.271325 24483 leader_election.cc:302] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ca289b211d458f9a35e6597ec281c1; no voters:
I20260807 19:50:03.272122 24483 leader_election.cc:288] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:50:03.272382 24493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:50:03.275458 24483 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:03.283430 24493 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [term 1 LEADER]: Becoming Leader. State: Replica: c1ca289b211d458f9a35e6597ec281c1, State: Running, Role: LEADER
I20260807 19:50:03.284440 24493 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } }
I20260807 19:50:03.288369 24495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ca289b211d458f9a35e6597ec281c1. Latest consensus state: current_term: 1 leader_uuid: "c1ca289b211d458f9a35e6597ec281c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } } }
I20260807 19:50:03.289197 24495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:03.289956 24494 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ca289b211d458f9a35e6597ec281c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ca289b211d458f9a35e6597ec281c1" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 36637 } } }
I20260807 19:50:03.290361 24494 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:03.290593 24506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:50:03.305979 24506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:50:03.318025 24506 catalog_manager.cc:1347] Generated new cluster ID: cfd10ce41fb7468280598cc59504e96c
I20260807 19:50:03.318234 24506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:50:03.331406 24506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:50:03.332907 24506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:50:03.348721 24506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ca289b211d458f9a35e6597ec281c1: Generated new TSK 0
I20260807 19:50:03.349792 24506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:50:03.369907 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24409
W20260807 19:50:03.371184 24478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47691
W20260807 19:50:03.372200 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24409: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:50:03.372380 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24409
2026-08-07T19:50:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:50:03 dist-test-slave-btw8 krb5kdc[24514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:50:03 dist-test-slave-btw8 krb5kdc[24514](info): set up 2 sockets
Aug 07 19:50:03 dist-test-slave-btw8 krb5kdc[24514](info): commencing operation
krb5kdc: starting...
W20260807 19:50:04.700155 795 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.002s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:50:04 dist-test-slave-btw8 krb5kdc[24514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:50:05.116773 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:41945
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:50:05.515046 24529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:05.515523 24529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:05.515678 24529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:05.523902 24529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:05.524075 24529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:05.524133 24529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:05.524197 24529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:05.524250 24529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:05.535727 24529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:05.537245 24529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:05.538949 24529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:05.553516 24537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:05.553608 24535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:05.553561 24534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:05.554607 24529 server_base.cc:1034] running on GCE node
I20260807 19:50:05.555579 24529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:05.557360 24529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:05.558696 24529 hybrid_clock.cc:648] HybridClock initialized: now 1786132205558554 us; error 60 us; skew 500 ppm
I20260807 19:50:05.559190 24529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:05 dist-test-slave-btw8 krb5kdc[24514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132205, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:05.563232 24529 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:50:05.565364 24529 webserver.cc:460] Webserver started at http://127.0.198.254:37179/ using document root <none> and password file <none>
I20260807 19:50:05.566215 24529 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:05.566381 24529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:05.566772 24529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:05.570385 24529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "87b6efe65bc446e69629f5909930e9ed"
format_stamp: "Formatted at 2026-08-07 19:50:05 on dist-test-slave-btw8"
server_key: "1bbe9915102328976d4a2194ac19fd8d"
server_key_iv: "9abf5efeb69e9a7cac1be9698ffcaf3b"
server_key_version: "encryptionkey@0"
I20260807 19:50:05.571482 24529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "87b6efe65bc446e69629f5909930e9ed"
format_stamp: "Formatted at 2026-08-07 19:50:05 on dist-test-slave-btw8"
server_key: "1bbe9915102328976d4a2194ac19fd8d"
server_key_iv: "9abf5efeb69e9a7cac1be9698ffcaf3b"
server_key_version: "encryptionkey@0"
I20260807 19:50:05.578114 24529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:50:05.582877 24544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:05.584645 24529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:50:05.584889 24529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "87b6efe65bc446e69629f5909930e9ed"
format_stamp: "Formatted at 2026-08-07 19:50:05 on dist-test-slave-btw8"
server_key: "1bbe9915102328976d4a2194ac19fd8d"
server_key_iv: "9abf5efeb69e9a7cac1be9698ffcaf3b"
server_key_version: "encryptionkey@0"
I20260807 19:50:05.585124 24529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:05.592255 24529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:05.597287 24529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:05.597637 24529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:05.619650 24529 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:41945
I20260807 19:50:05.619673 24600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:41945 every 8 connection(s)
I20260807 19:50:05.621295 24529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:50:05.625560 24601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:05.630551 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24529
I20260807 19:50:05.630775 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:50:05.631392 795 external_mini_cluster.cc:1428] Setting key 3194b33f3a0902bd47600bbe8633d7a7
I20260807 19:50:05.640069 24601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed: Bootstrap starting.
I20260807 19:50:05.642725 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:05.633550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52449 (local address 127.0.198.254:41945)
0807 19:50:05.634584 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:05.634597 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:05.634645 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:50:05.637876 (+ 3231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:05.637912 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:05.638598 (+ 686us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:05.639045 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:05.640797 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:05.640804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:05.640819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:50:05.640849 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:05.640931 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:50:05.640957 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:05.641210 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:05.641339 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:50:05.641631 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":538,"threads_started":1}
I20260807 19:50:05.644591 24601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:05.646066 24601 log.cc:824] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:05.647492 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:50:05.650364 24601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed: No bootstrap required, opened a new log
I20260807 19:50:05.657102 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:05.653753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37097 (local address 127.0.198.254:41945)
0807 19:50:05.653989 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:05.653996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:05.654553 (+ 557us) server_negotiation.cc:407] Connection header received
0807 19:50:05.654739 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:05.654745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:05.654823 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:05.654966 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:05.656153 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:05.656164 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:05.656170 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:50:05.656196 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:05.656258 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:50:05.656284 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:05.656444 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:05.656624 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 19:50:05.656714 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:50:05.661247 24601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } }
I20260807 19:50:05.661693 24601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:50:05.661829 24601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b6efe65bc446e69629f5909930e9ed, State: Initialized, Role: FOLLOWER
I20260807 19:50:05.662576 24601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } }
I20260807 19:50:05.662832 24601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:50:05.663030 24601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:50:05.663246 24601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:05.665783 24601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } }
I20260807 19:50:05.666409 24601 leader_election.cc:302] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87b6efe65bc446e69629f5909930e9ed; no voters:
I20260807 19:50:05.667155 24601 leader_election.cc:288] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:50:05.667335 24614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:50:05.668371 24614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [term 1 LEADER]: Becoming Leader. State: Replica: 87b6efe65bc446e69629f5909930e9ed, State: Running, Role: LEADER
I20260807 19:50:05.669039 24614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } }
I20260807 19:50:05.669726 24601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:05.672634 24616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b6efe65bc446e69629f5909930e9ed. Latest consensus state: current_term: 1 leader_uuid: "87b6efe65bc446e69629f5909930e9ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } } }
I20260807 19:50:05.673187 24616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:05.672623 24615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87b6efe65bc446e69629f5909930e9ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b6efe65bc446e69629f5909930e9ed" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 41945 } } }
I20260807 19:50:05.673743 24615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:05.678614 24621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:50:05.683908 24621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:50:05.696445 24621 catalog_manager.cc:1347] Generated new cluster ID: 7b1b50a70f064fd79cca5119a231e5f2
I20260807 19:50:05.696607 24621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:50:05.755856 24621 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:50:05.756492 24621 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:50:05.770634 24621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87b6efe65bc446e69629f5909930e9ed: Generated new TSK 0
I20260807 19:50:05.771940 24621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:50:05.792761 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24529
W20260807 19:50:05.794142 24596 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52449
W20260807 19:50:05.795017 795 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24529: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:50:05.795223 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24529
2026-08-07T19:50:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2411 ms)
[----------] 18 tests from AuthTokenIssuingTest (42651 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:50:05 dist-test-slave-btw8 krb5kdc[24636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:50:05 dist-test-slave-btw8 krb5kdc[24636](info): set up 2 sockets
Aug 07 19:50:05 dist-test-slave-btw8 krb5kdc[24636](info): commencing operation
krb5kdc: starting...
W20260807 19:50:07.050565 795 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.004s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:50:07 dist-test-slave-btw8 krb5kdc[24636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:50:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:50:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:50:07.457295 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34765
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:50:07.853210 24649 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:07.853683 24649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:07.853819 24649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:07.853960 24649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:07.861732 24649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:07.861909 24649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:07.862018 24649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:07.862109 24649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:07.862203 24649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:07.873119 24649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:07.874627 24649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:07.876204 24649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:07.889810 24654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:07.889873 24657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:07.889853 24655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:07.890833 24649 server_base.cc:1034] running on GCE node
I20260807 19:50:07.891861 24649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:07.893512 24649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:07.894824 24649 hybrid_clock.cc:648] HybridClock initialized: now 1786132207894716 us; error 48 us; skew 500 ppm
I20260807 19:50:07.895256 24649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:07 dist-test-slave-btw8 krb5kdc[24636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:07.905699 24649 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:50:07.907994 24649 webserver.cc:460] Webserver started at http://127.0.198.254:46593/ using document root <none> and password file <none>
I20260807 19:50:07.908819 24649 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:07.908942 24649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:07.909425 24649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:07.913635 24649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "95fc56a4680a41739c98028fc49d2b38"
format_stamp: "Formatted at 2026-08-07 19:50:07 on dist-test-slave-btw8"
server_key: "8638b84b04ceb634132dd115714a4a5d"
server_key_iv: "9bbdb32c0ffa8b4db41f7745dd15829e"
server_key_version: "encryptionkey@0"
I20260807 19:50:07.914860 24649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "95fc56a4680a41739c98028fc49d2b38"
format_stamp: "Formatted at 2026-08-07 19:50:07 on dist-test-slave-btw8"
server_key: "8638b84b04ceb634132dd115714a4a5d"
server_key_iv: "9bbdb32c0ffa8b4db41f7745dd15829e"
server_key_version: "encryptionkey@0"
I20260807 19:50:07.922006 24649 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260807 19:50:07.926721 24664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:07.928714 24649 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:50:07.928995 24649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "95fc56a4680a41739c98028fc49d2b38"
format_stamp: "Formatted at 2026-08-07 19:50:07 on dist-test-slave-btw8"
server_key: "8638b84b04ceb634132dd115714a4a5d"
server_key_iv: "9bbdb32c0ffa8b4db41f7745dd15829e"
server_key_version: "encryptionkey@0"
I20260807 19:50:07.929308 24649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:07.954289 24649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:07.960522 24649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:07.960951 24649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:07.987241 24649 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:34765
I20260807 19:50:07.987272 24718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:34765 every 8 connection(s)
I20260807 19:50:07.988865 24649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:50:07.989482 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24649
I20260807 19:50:07.989791 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:50:07.990703 795 external_mini_cluster.cc:1428] Setting key ac1292612ee49c1e3907fb3f5b606077
I20260807 19:50:07.993520 24719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:08.004969 24719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:08 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.253@KRBTEST.COM; defaulting to no policy
I20260807 19:50:08.033054 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.009448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34765 (local address 127.0.0.1:49652)
0807 19:50:08.010333 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.010384 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:08.012733 (+ 2349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:08.013712 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:08.013720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:08.014846 (+ 1126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:08.015301 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.015402 (+ 101us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.017307 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.017313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.018736 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.018752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.019921 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.019928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.020159 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.021062 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:08.021096 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:08.024178 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:08.028343 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.028361 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.028372 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.028827 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.029295 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.029301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.029304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.029478 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.030319 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:08.030329 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:08.030542 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 19:50:08.030806 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 19:50:08.031561 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":222,"thread_start_us":371,"threads_started":1}
I20260807 19:50:08.033073 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.010707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.0.198.254:34765)
0807 19:50:08.011467 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:08.011476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:08.011535 (+ 59us) server_negotiation.cc:407] Connection header received
0807 19:50:08.013170 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:08.013177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:08.013463 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:08.013690 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:08.015652 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.017096 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.018948 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.019654 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.019886 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.024507 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:08.024518 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:08.024533 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:08.024578 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:08.028149 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.028987 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.028994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.028996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.029169 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.029595 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.029606 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.029608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.030159 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:08.030392 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:08.030917 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 19:50:08.031431 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":487,"thread_start_us":314,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.253@KRBTEST.COM" created.
I20260807 19:50:08.037395 24719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:08.039165 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.038456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51046)
0807 19:50:08.038783 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.038965 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260807 19:50:08.039727 24667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
I20260807 19:50:08.042187 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.041456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51052)
0807 19:50:08.041790 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.041921 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 19:50:08.042538 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:41637: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
I20260807 19:50:08.081638 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:41637
--webserver_interface=127.0.198.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:50:08.091979 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } attempt: 1
I20260807 19:50:08.093465 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.092831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51068)
0807 19:50:08.093099 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.093248 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 19:50:08.094807 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.094311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51072)
0807 19:50:08.094507 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.094607 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:50:08.095060 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:41637: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
I20260807 19:50:08.163461 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } attempt: 2
I20260807 19:50:08.164834 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.164304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51078)
0807 19:50:08.164584 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.164684 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:50:08.166090 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.165668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51092)
0807 19:50:08.165828 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.165930 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:50:08.166299 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:41637: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
I20260807 19:50:08.257635 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } attempt: 3
I20260807 19:50:08.259183 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.258524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51094)
0807 19:50:08.258904 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.259023 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260807 19:50:08.260572 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.260048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51098)
0807 19:50:08.260296 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.260393 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260807 19:50:08.260824 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:41637: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
I20260807 19:50:08.404142 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } attempt: 4
I20260807 19:50:08.405444 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.404868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51100)
0807 19:50:08.405207 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.405299 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 19:50:08.406589 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.406213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51102)
0807 19:50:08.406377 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.406450 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:50:08.406783 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:41637: Network error: Client connection negotiation failed: client connection to 127.0.198.253:41637: connect: Connection refused (error 111)
W20260807 19:50:08.491425 24727 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:08.491874 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:08.491998 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:08.492129 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:08.500262 24727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:08.500409 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:08.500527 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:08.500612 24727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:08.500783 24727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:08.511641 24727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.198.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:08.513193 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:08.514921 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:08.529399 24735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:08.529393 24733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:08.529399 24732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:08.531162 24727 server_base.cc:1034] running on GCE node
I20260807 19:50:08.532160 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:08.533921 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:08.535442 24727 hybrid_clock.cc:648] HybridClock initialized: now 1786132208535257 us; error 96 us; skew 500 ppm
I20260807 19:50:08.535890 24727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:08 dist-test-slave-btw8 krb5kdc[24636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:08.539989 24727 init.cc:377] Logged in from keytab as kudu/127.0.198.253@KRBTEST.COM (short username kudu)
I20260807 19:50:08.541872 24727 webserver.cc:460] Webserver started at http://127.0.198.253:35043/ using document root <none> and password file <none>
I20260807 19:50:08.542685 24727 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:08.542835 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:08.543221 24727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:08.546543 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data/instance:
uuid: "908770844ffc4f3589fe5cd94f506a19"
format_stamp: "Formatted at 2026-08-07 19:50:08 on dist-test-slave-btw8"
server_key: "dfe0f8d032eab69a8f2be0e64aefd3d1"
server_key_iv: "091139086686884c3ed20b2a0580f216"
server_key_version: "encryptionkey@0"
I20260807 19:50:08.547576 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal/instance:
uuid: "908770844ffc4f3589fe5cd94f506a19"
format_stamp: "Formatted at 2026-08-07 19:50:08 on dist-test-slave-btw8"
server_key: "dfe0f8d032eab69a8f2be0e64aefd3d1"
server_key_iv: "091139086686884c3ed20b2a0580f216"
server_key_version: "encryptionkey@0"
I20260807 19:50:08.553965 24727 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:50:08.558977 24742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:08.560914 24727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:50:08.561183 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "908770844ffc4f3589fe5cd94f506a19"
format_stamp: "Formatted at 2026-08-07 19:50:08 on dist-test-slave-btw8"
server_key: "dfe0f8d032eab69a8f2be0e64aefd3d1"
server_key_iv: "091139086686884c3ed20b2a0580f216"
server_key_version: "encryptionkey@0"
I20260807 19:50:08.561474 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:08.584062 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:08.589180 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:08.589514 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:08.612599 24727 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.253:41637
I20260807 19:50:08.612636 24797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.253:41637 every 8 connection(s)
I20260807 19:50:08.614204 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
I20260807 19:50:08.614869 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24727
I20260807 19:50:08.615150 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-1/wal/instance
I20260807 19:50:08.615882 795 external_mini_cluster.cc:1428] Setting key f5cad2fa18c09cb0a501cacc60c5f9fb
I20260807 19:50:08.618983 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:08.629904 24798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:08 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.198.252@KRBTEST.COM; defaulting to no policy
I20260807 19:50:08.657123 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.633352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.0.198.254:34765)
0807 19:50:08.634024 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:08.634034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:08.635609 (+ 1575us) server_negotiation.cc:407] Connection header received
0807 19:50:08.637932 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:08.637942 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:08.638064 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:08.638207 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:08.640213 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.641477 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.643328 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.644162 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.644317 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.648885 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:08.648896 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:08.648904 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:08.648965 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:08.653194 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.654232 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.654241 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.654246 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.654422 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.655021 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.655027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.655032 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.655510 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:08.655686 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:08.656635 (+ 949us) server_negotiation.cc:299] Negotiation successful
0807 19:50:08.656841 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":390,"threads_started":1}
I20260807 19:50:08.658275 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.634255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34765 (local address 127.0.0.1:49662)
0807 19:50:08.635337 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.635437 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:08.637559 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:08.638481 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:08.638491 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:08.639166 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:08.639956 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.639976 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.641704 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.641708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.643148 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.643163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.644454 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.644461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.644721 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.645560 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:08.645592 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:08.648507 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:08.653411 (+ 4904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.653420 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.653436 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.654050 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.654609 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.654614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.654617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.654866 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.655715 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:08.655722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:08.656138 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 19:50:08.656404 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 19:50:08.656916 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":459,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.252@KRBTEST.COM" created.
I20260807 19:50:08.660501 24798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:08 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:08.676927 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.661488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51114)
0807 19:50:08.661763 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.661781 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:08.661893 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:08.663110 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:08.663114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:08.663680 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:08.663830 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.663839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.665740 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.665748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.666814 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.666826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.667771 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.667777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.667943 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.668606 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:08.668625 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:08.670738 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:08.674227 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.674235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.674239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.674628 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.675011 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.675017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.675020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.675169 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.675867 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:08.675878 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:08.676029 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 19:50:08.676176 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:50:08.676491 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:50:08.677044 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.661818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.0.198.253:41637)
0807 19:50:08.662592 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:08.662599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:08.662628 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:50:08.662679 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:08.662685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:08.662933 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:08.663136 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:08.664027 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.665580 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.666950 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.667596 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.667750 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.670964 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:08.670978 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:08.670994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:08.671029 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:08.674081 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.674775 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.674783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.674785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.674900 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.675277 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.675285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.675288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.675732 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:08.675968 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:08.676336 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 19:50:08.676638 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":602,"thread_start_us":433,"threads_started":1}
I20260807 19:50:08.679996 24798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:08.681641 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.680870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36734)
0807 19:50:08.681118 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.681324 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 19:50:08.682104 24745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.684391 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.683851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36742)
0807 19:50:08.684067 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.684193 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:50:08.684690 24798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
I20260807 19:50:08.703293 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:44181
--webserver_interface=127.0.198.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:50:08.706046 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } attempt: 5
Aug 07 19:50:08 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:08.722903 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.707051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.0.198.253:41637)
0807 19:50:08.707289 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:08.707297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:08.707326 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:50:08.707492 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:08.707499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:08.707596 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:08.707722 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:08.708605 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.709739 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.711159 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.712012 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.712160 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.715647 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:08.715657 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:08.715672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:08.715733 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:08.719439 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.720514 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.720519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.720521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.720750 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:08.721260 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:08.721264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:08.721266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:08.721579 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:08.721720 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:08.722349 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 19:50:08.722600 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:50:08.723079 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.706931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51118)
0807 19:50:08.707157 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.707186 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:08.707303 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:08.707786 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:08.707790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:08.708230 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:08.708437 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.708446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.709908 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.709913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.710962 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:08.710975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:08.712207 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:08.712211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:08.712407 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:08.713097 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:08.713116 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:08.715315 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:08.719624 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.719633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.719692 (+ 59us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.720341 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.720905 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:08.720912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:08.720919 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:08.721115 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:08.721734 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:08.721742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:08.721953 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 19:50:08.722161 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 19:50:08.722527 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:50:08.724268 24719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:08.725683 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.724969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36754)
0807 19:50:08.725419 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.725510 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:50:08.726843 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.726428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36756)
0807 19:50:08.726598 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.726732 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:50:08.727059 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.734130 24798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 1
I20260807 19:50:08.735641 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.734957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36772)
0807 19:50:08.735280 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.735461 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:50:08.736923 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.736456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36782)
0807 19:50:08.736677 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.736779 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260807 19:50:08.737159 24798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.778321 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 1
I20260807 19:50:08.779783 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.779174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36784)
0807 19:50:08.779477 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.779594 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 19:50:08.781348 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.780821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36798)
0807 19:50:08.781009 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.781173 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 19:50:08.781638 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.805434 24798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 2
I20260807 19:50:08.806953 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.806167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36802)
0807 19:50:08.806444 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.806713 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 19:50:08.808681 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.808044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36810)
0807 19:50:08.808361 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.808466 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
W20260807 19:50:08.808990 24798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.849901 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 2
I20260807 19:50:08.851601 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.850910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36814)
0807 19:50:08.851213 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.851322 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 19:50:08.853094 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.852653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36822)
0807 19:50:08.852839 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.852933 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 19:50:08.853430 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.900343 24798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 3
I20260807 19:50:08.901731 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.901189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36824)
0807 19:50:08.901445 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.901552 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 19:50:08.903236 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.902665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36826)
0807 19:50:08.902965 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.903070 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260807 19:50:08.903542 24798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:08.959726 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 3
I20260807 19:50:08.961207 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.960654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36830)
0807 19:50:08.960895 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.961002 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:50:08.962567 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:08.962082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36842)
0807 19:50:08.962331 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:08.962428 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260807 19:50:08.962836 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
I20260807 19:50:09.046810 24798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 4
I20260807 19:50:09.048331 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.047697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36856)
0807 19:50:09.047997 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.048130 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 19:50:09.049439 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.049072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36866)
0807 19:50:09.049236 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.049321 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 19:50:09.049669 24798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
W20260807 19:50:09.116669 24807 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:09.117193 24807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:09.117341 24807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:09.117503 24807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:09.125982 24807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:09.126180 24807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:09.126302 24807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:09.126400 24807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:09.126495 24807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:09.138545 24807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34765,127.0.198.253:41637,127.0.198.252:44181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:44181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.198.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:09.140380 24807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:09.140168 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 4
I20260807 19:50:09.141961 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.141340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36876)
0807 19:50:09.141659 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.141769 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260807 19:50:09.142325 24807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:50:09.143494 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.142900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36878)
0807 19:50:09.143150 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.143263 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:50:09.143985 24719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:44181: Network error: Client connection negotiation failed: client connection to 127.0.198.252:44181: connect: Connection refused (error 111)
W20260807 19:50:09.157313 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:09.157701 24816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:09.158528 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:09.160362 24807 server_base.cc:1034] running on GCE node
I20260807 19:50:09.161638 24807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:09.163585 24807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:09.164898 24807 hybrid_clock.cc:648] HybridClock initialized: now 1786132209164822 us; error 59 us; skew 500 ppm
I20260807 19:50:09.165416 24807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:09.169538 24807 init.cc:377] Logged in from keytab as kudu/127.0.198.252@KRBTEST.COM (short username kudu)
I20260807 19:50:09.171435 24807 webserver.cc:460] Webserver started at http://127.0.198.252:46019/ using document root <none> and password file <none>
I20260807 19:50:09.172389 24807 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:09.172560 24807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:09.172941 24807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:09.177379 24807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data/instance:
uuid: "43021bd9f5d64e30bffabfa76c77688e"
format_stamp: "Formatted at 2026-08-07 19:50:09 on dist-test-slave-btw8"
server_key: "71a7337015dd7b32f4491965be5a766b"
server_key_iv: "084d0a0e2a8029b97dfef708ef8291cd"
server_key_version: "encryptionkey@0"
I20260807 19:50:09.178529 24807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal/instance:
uuid: "43021bd9f5d64e30bffabfa76c77688e"
format_stamp: "Formatted at 2026-08-07 19:50:09 on dist-test-slave-btw8"
server_key: "71a7337015dd7b32f4491965be5a766b"
server_key_iv: "084d0a0e2a8029b97dfef708ef8291cd"
server_key_version: "encryptionkey@0"
I20260807 19:50:09.185771 24807 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 19:50:09.190227 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:09.192006 24807 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 19:50:09.192258 24807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "43021bd9f5d64e30bffabfa76c77688e"
format_stamp: "Formatted at 2026-08-07 19:50:09 on dist-test-slave-btw8"
server_key: "71a7337015dd7b32f4491965be5a766b"
server_key_iv: "084d0a0e2a8029b97dfef708ef8291cd"
server_key_version: "encryptionkey@0"
I20260807 19:50:09.192535 24807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:09.218990 24807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:09.224731 24807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:09.225062 24807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:09.248502 24807 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.252:44181
I20260807 19:50:09.248541 24878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.252:44181 every 8 connection(s)
I20260807 19:50:09.250245 24807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
I20260807 19:50:09.254977 24879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:09.257742 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 24807
I20260807 19:50:09.257972 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131811070045-795-0/minicluster-data/master-2/wal/instance
I20260807 19:50:09.258657 795 external_mini_cluster.cc:1428] Setting key 5b8d195a3ff75118de63334f94705c41
I20260807 19:50:09.267272 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:09.288616 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.261113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.0.198.252:44181)
0807 19:50:09.262319 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.262341 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.262406 (+ 65us) server_negotiation.cc:407] Connection header received
0807 19:50:09.264319 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.264354 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.264858 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.265252 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.267535 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.269026 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.270748 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.272179 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.272350 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.277802 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.277816 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.277910 (+ 94us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.277996 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.282961 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.284149 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.284159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.284168 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.284409 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.284989 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.284994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.284996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.285681 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.286054 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.286622 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.287127 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"spinlock_wait_cycles":22912,"thread_start_us":591,"threads_started":1}
I20260807 19:50:09.290556 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.270550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34765 (local address 127.0.0.1:49676)
0807 19:50:09.271678 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.271722 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:09.271933 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:09.272636 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:09.272647 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:09.273586 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:09.273863 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.273878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.275656 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.275660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.276872 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.276888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.278032 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.278039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.278317 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.279415 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:09.279454 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:09.282961 (+ 3507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:09.287545 (+ 4584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.287555 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.287563 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.288152 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.288713 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.288719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.288722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.288889 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.289519 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:09.289527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:09.289758 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 19:50:09.289961 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 19:50:09.290236 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":621,"threads_started":1}
I20260807 19:50:09.290912 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.270536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.0.198.254:34765)
0807 19:50:09.271205 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.271212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.271822 (+ 610us) server_negotiation.cc:407] Connection header received
0807 19:50:09.272073 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.272078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.272233 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.272410 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.274117 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.275475 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.277067 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.277761 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.277891 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.283309 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.283319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.283326 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.283405 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.287342 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.288328 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.288333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.288336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.288572 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.289053 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.289058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.289060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.289384 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.289530 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.290116 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.290403 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":335,"threads_started":1}
I20260807 19:50:09.292377 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:09.310242 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.293078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:41637 (local address 127.0.0.1:51120)
0807 19:50:09.293237 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.293256 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:09.293355 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:09.294169 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:09.294172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:09.294521 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:09.294718 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.294732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.296120 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.296124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.297180 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.297195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.298211 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.298216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.298365 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.299155 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:09.299174 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:09.302062 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:09.306891 (+ 4829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.306896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.306900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.307534 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.308054 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.308060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.308064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.308283 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.309214 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:09.309221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:09.309390 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 19:50:09.309516 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:50:09.309770 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:50:09.310598 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.293177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.0.198.253:41637)
0807 19:50:09.293785 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.293792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.293814 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:50:09.293882 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.293890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.293984 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.294126 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.294923 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.295936 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.297352 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.298040 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.298177 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.302436 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.302442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.302446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.302491 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.306665 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.307736 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.307741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.307744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.307909 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.308437 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.308443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.308446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.308931 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.309087 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.309687 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.309969 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":315,"threads_started":1}
I20260807 19:50:09.311709 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.292209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.0.198.254:34765)
0807 19:50:09.292367 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.292371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.292624 (+ 253us) server_negotiation.cc:407] Connection header received
0807 19:50:09.292799 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.292809 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.292881 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.292987 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.294757 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.295620 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.297190 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.297809 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.297949 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.303420 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.303431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.303439 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.303490 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.307291 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.308328 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.308334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.308337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.308571 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.309291 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.309295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.309297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.309821 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.310252 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.311010 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.311320 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:50:09.311847 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:09.328953 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.312711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36890)
0807 19:50:09.312929 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.312950 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:09.313073 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:09.313440 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:09.313447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:09.314115 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:09.314274 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.314284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.315470 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.315475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.316433 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.316446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.317187 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.317193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.317422 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.318251 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:09.318270 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:09.321013 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:09.324767 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.324773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.324777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.325189 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.325798 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.325806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.325812 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.326009 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.327275 (+ 1266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:09.327282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:09.327705 (+ 423us) client_negotiation.cc:769] Sending connection context
0807 19:50:09.327948 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 19:50:09.328428 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:50:09.329605 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.312903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.0.198.252:44181)
0807 19:50:09.313074 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.313080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.313107 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:50:09.313163 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.313168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.313241 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.313347 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.314422 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.315147 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.316588 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.317001 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.317140 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.321273 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.321279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.321283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.321319 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.324578 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.325330 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.325336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.325339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.325507 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.326297 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.326305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.326307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.327040 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.327225 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.328185 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.328981 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":47}
I20260807 19:50:09.333117 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.313788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.0.198.253:41637)
0807 19:50:09.313963 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.313969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.313986 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:50:09.314150 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.314155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.314228 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.314328 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.316130 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.316907 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.318384 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.318910 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.319073 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.324854 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.324860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.324863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.324903 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.329546 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.330591 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.330599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.330603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.330769 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.331343 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.331395 (+ 52us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.331400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.331841 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.332165 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.332423 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.332710 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:50:09.337167 24879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: Bootstrap starting.
I20260807 19:50:09.341084 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:50:09.342084 24879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:09.343605 24879 log.cc:824] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:09.347820 24879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: No bootstrap required, opened a new log
I20260807 19:50:09.348917 24798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 5
I20260807 19:50:09.345613 24693 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "95fc56a4680a41739c98028fc49d2b38"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:50:09.360072 24879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.360502 24879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43021bd9f5d64e30bffabfa76c77688e, State: Initialized, Role: FOLLOWER
I20260807 19:50:09.361132 24879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.362614 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.363137 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.364053 24879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:09.369059 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.349865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.0.198.252:44181)
0807 19:50:09.350498 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.350510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.350529 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:50:09.350591 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.350595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.350675 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.350782 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.351993 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.353307 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.355206 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.355856 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.356022 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.361064 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.361071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.361075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.361120 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.365234 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.366234 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.366242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.366247 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.366458 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.367010 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.367017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.367021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.367631 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.367843 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.368378 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.368635 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465}
I20260807 19:50:09.369074 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.349669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36904)
0807 19:50:09.349984 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.350009 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:09.350165 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:09.350874 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:09.350882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:09.351494 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:09.351751 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.351767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.353499 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.353506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.355006 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.355039 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.356056 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.356064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.356352 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.357175 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:09.357207 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:09.360782 (+ 3575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:09.365442 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.365449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.365454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.366090 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.366615 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.366623 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.366629 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.366864 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.367793 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:09.367801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:09.368041 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 19:50:09.368224 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 19:50:09.368612 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260807 19:50:09.375038 24798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: Bootstrap starting.
I20260807 19:50:09.379621 24798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: Neither blocks nor log segments found. Creating new log.
W20260807 19:50:09.380609 24900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:50:09.380748 24900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:50:09.381135 24798 log.cc:824] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:09.384932 24798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: No bootstrap required, opened a new log
I20260807 19:50:09.378453 24807 master_runner.cc:186] Error getting master registration for 127.0.198.254:34765: OK, instance_id { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" instance_seqno: 1786132207979868 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:50:09.389043 24807 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:50:09.397912 24798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.398437 24798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908770844ffc4f3589fe5cd94f506a19, State: Initialized, Role: FOLLOWER
I20260807 19:50:09.399082 24798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.400508 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.401000 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.402176 24798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:50:09.414980 24918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:50:09.415416 24918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:50:09.421248 24719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } attempt: 5
I20260807 19:50:09.411263 24727 master_runner.cc:186] Error getting master registration for 127.0.198.254:34765: OK, instance_id { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" instance_seqno: 1786132207979868 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:50:09.427119 24727 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 19:50:09 dist-test-slave-btw8 krb5kdc[24636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:50:09.443949 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.423253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.0.198.252:44181)
0807 19:50:09.423509 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.423518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.423546 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:50:09.423616 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.423622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.423755 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.423891 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.425045 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.426289 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.428495 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.429313 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.429470 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.434630 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.434641 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.434652 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.434702 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.439123 (+ 4421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.440865 (+ 1742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.440873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.440877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.441079 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.441865 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.441871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.441874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.442312 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.442491 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.443217 (+ 726us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.443548 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:50:09.444591 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.422162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:44181 (local address 127.0.0.1:36912)
0807 19:50:09.422719 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:09.422745 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:09.422899 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:09.423958 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:09.423966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:09.424592 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:09.424854 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.424873 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.426500 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.426510 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.428289 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:09.428314 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.429560 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.429568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:09.429831 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.430663 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:09.430693 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:09.434258 (+ 3565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:09.439465 (+ 5207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.439473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.439479 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.440648 (+ 1169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.441245 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:09.441275 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:09.441279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:09.441672 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:09.442457 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:09.442463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:09.442791 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 19:50:09.443155 (+ 364us) client_negotiation.cc:240] Negotiation successful
0807 19:50:09.443780 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394}
I20260807 19:50:09.451807 24719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38: Bootstrap starting.
I20260807 19:50:09.456804 24719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:09.458600 24719 log.cc:824] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:09.462920 24719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38: No bootstrap required, opened a new log
I20260807 19:50:09.472554 24719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.473151 24719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fc56a4680a41739c98028fc49d2b38, State: Initialized, Role: FOLLOWER
I20260807 19:50:09.473830 24719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.475482 24922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.476423 24922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.476888 24719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:09.477101 24693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:09.477332 24693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:09.481015 24693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
W20260807 19:50:09.483193 24830 tablet.cc:2369] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:50:09.484195 24772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95fc56a4680a41739c98028fc49d2b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "908770844ffc4f3589fe5cd94f506a19"
I20260807 19:50:09.484779 24772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:09.488685 24772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc56a4680a41739c98028fc49d2b38 in term 1.
I20260807 19:50:09.489732 24667 leader_election.cc:302] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908770844ffc4f3589fe5cd94f506a19, 95fc56a4680a41739c98028fc49d2b38; no voters:
I20260807 19:50:09.492412 24853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95fc56a4680a41739c98028fc49d2b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "43021bd9f5d64e30bffabfa76c77688e"
I20260807 19:50:09.492946 24853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:09.496832 24853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95fc56a4680a41739c98028fc49d2b38 in term 1.
I20260807 19:50:09.497686 24693 leader_election.cc:288] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [CANDIDATE]: Term 1 election: Requested vote from peers 908770844ffc4f3589fe5cd94f506a19 (127.0.198.253:41637), 43021bd9f5d64e30bffabfa76c77688e (127.0.198.252:44181)
I20260807 19:50:09.498953 24922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:50:09.506234 24922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [term 1 LEADER]: Becoming Leader. State: Replica: 95fc56a4680a41739c98028fc49d2b38, State: Running, Role: LEADER
I20260807 19:50:09.507105 24922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } }
I20260807 19:50:09.515740 24927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fc56a4680a41739c98028fc49d2b38. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.516584 24927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:09.523257 24933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:50:09.547616 24933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:50:09.548346 24944 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:50:09.587191 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.539026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.0.198.253:41637)
0807 19:50:09.539540 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.539549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.548042 (+ 8493us) server_negotiation.cc:407] Connection header received
0807 19:50:09.548398 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.548406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.548513 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.548615 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.559510 (+ 10895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.560663 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.562913 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.563685 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.563818 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.568097 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.568104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.568109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.568152 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.572212 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.584162 (+ 11950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.584170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.584174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.584409 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.585070 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.585077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.585080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.585537 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.585858 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.586504 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.586766 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342}
I20260807 19:50:09.587198 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.539019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.0.198.254:34765)
0807 19:50:09.539313 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.539324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.544039 (+ 4715us) server_negotiation.cc:407] Connection header received
0807 19:50:09.544740 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.544748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.544847 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.544936 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.548822 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.550166 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.552899 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.554008 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.554203 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.571645 (+ 17442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.571658 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.571664 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.571760 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.576015 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.577201 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.577208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.577212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.577471 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.578047 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.578064 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.578067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.578720 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.586162 (+ 7442us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.586390 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.586709 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260807 19:50:09.592669 24749 tablet.cc:2369] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:50:09.604234 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:09.564062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.0.198.252:44181)
0807 19:50:09.564345 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:09.564353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:09.580488 (+ 16135us) server_negotiation.cc:407] Connection header received
0807 19:50:09.580587 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:09.580594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:09.580691 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:09.580806 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:09.582748 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.583870 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.589427 (+ 5557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:09.590148 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:09.590301 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:09.595473 (+ 5172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:09.595482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:09.595487 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:09.595538 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:09.599727 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.600830 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.600836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.600840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.601009 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:09.601517 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:09.601524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:09.601528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:09.601959 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:09.602223 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:09.603505 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0807 19:50:09.603801 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:50:09.604807 24853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [term 1 FOLLOWER]: Refusing update from remote peer 95fc56a4680a41739c98028fc49d2b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:50:09.606267 24927 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:50:09.608559 24772 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [term 1 FOLLOWER]: Refusing update from remote peer 95fc56a4680a41739c98028fc49d2b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:50:09.614706 24927 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:50:09.641193 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fc56a4680a41739c98028fc49d2b38. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.641672 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.648926 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fc56a4680a41739c98028fc49d2b38. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.649307 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.654568 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.654906 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.666409 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.666750 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:09.676789 24933 catalog_manager.cc:1347] Generated new cluster ID: 55db7512ede64344b96997a3d1e8a1a5
I20260807 19:50:09.676932 24933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:50:09.685904 24927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.686290 24927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:09.686704 24927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95fc56a4680a41739c98028fc49d2b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95fc56a4680a41739c98028fc49d2b38" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34765 } } peers { permanent_uuid: "908770844ffc4f3589fe5cd94f506a19" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 41637 } } peers { permanent_uuid: "43021bd9f5d64e30bffabfa76c77688e" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 44181 } } }
I20260807 19:50:09.687017 24927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:09.745544 24933 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:50:09.747825 24933 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:50:09.771312 24933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95fc56a4680a41739c98028fc49d2b38: Generated new TSK 0
I20260807 19:50:09.772660 24933 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:50:09.961539 795 mini_kdc.cc:348] Time spent kdestroy: real 0.178s user 0.000s sys 0.037s
W20260807 19:50:10.035993 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.019340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.0.198.253:41637)
0807 19:50:10.019592 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.019599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.032826 (+ 13227us) server_negotiation.cc:407] Connection header received
0807 19:50:10.033825 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.033835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.033955 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.034045 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.035613 (+ 1568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51160: BlockingRecv error: recv got EOF from 127.0.0.1:51160 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 19:50:10.037994 24884 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.019128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.0.198.254:34765)
0807 19:50:10.026342 (+ 7214us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.026354 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.032147 (+ 5793us) server_negotiation.cc:407] Connection header received
0807 19:50:10.033350 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.033359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.033458 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.033542 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.037731 (+ 4189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49706: BlockingRecv error: recv got EOF from 127.0.0.1:49706 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 19:50:10.040858 24883 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.033024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.0.198.252:44181)
0807 19:50:10.033259 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.033311 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.038881 (+ 5570us) server_negotiation.cc:407] Connection header received
0807 19:50:10.039068 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.039073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.039162 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.039279 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.040588 (+ 1309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36924: BlockingRecv error: recv got EOF from 127.0.0.1:36924 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 19:50:10.111733 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.094417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.0.198.253:41637)
0807 19:50:10.094680 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.094689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.108752 (+ 14063us) server_negotiation.cc:407] Connection header received
0807 19:50:10.109537 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.109548 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.109723 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.109835 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.111374 (+ 1539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51170: BlockingRecv error: recv got EOF from 127.0.0.1:51170 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:50:10.161998 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.149887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.0.198.253:41637)
0807 19:50:10.150126 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.150136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.159508 (+ 9372us) server_negotiation.cc:407] Connection header received
0807 19:50:10.160172 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.160180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.160303 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.160389 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.161689 (+ 1300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51180: BlockingRecv error: recv got EOF from 127.0.0.1:51180 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:50:10.213014 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.200562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.0.198.253:41637)
0807 19:50:10.200830 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.200840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.210760 (+ 9920us) server_negotiation.cc:407] Connection header received
0807 19:50:10.210972 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.210980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.211109 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.211245 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.212682 (+ 1437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51188: BlockingRecv error: recv got EOF from 127.0.0.1:51188 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260807 19:50:10.275658 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.269424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.0.198.253:41637)
0807 19:50:10.269668 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.269675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.273340 (+ 3665us) server_negotiation.cc:407] Connection header received
0807 19:50:10.273661 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.273689 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.273796 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.273872 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.275319 (+ 1447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51194: BlockingRecv error: recv got EOF from 127.0.0.1:51194 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260807 19:50:10.297683 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.292658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.0.198.253:41637)
0807 19:50:10.292952 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.292962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.296047 (+ 3085us) server_negotiation.cc:407] Connection header received
0807 19:50:10.296298 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.296303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.296381 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.296507 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.297422 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51206: BlockingRecv error: recv got EOF from 127.0.0.1:51206 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 19:50:10.320463 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.317688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.0.198.253:41637)
0807 19:50:10.318005 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.318016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.318671 (+ 655us) server_negotiation.cc:407] Connection header received
0807 19:50:10.318952 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.318957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.319037 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.319158 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.320092 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51218: BlockingRecv error: recv got EOF from 127.0.0.1:51218 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 19:50:10.359555 24886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:10.356829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.0.198.253:41637)
0807 19:50:10.357096 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.357107 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.357712 (+ 605us) server_negotiation.cc:407] Connection header received
0807 19:50:10.357979 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.357984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.358112 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.358245 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:10.359205 (+ 960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51220: BlockingRecv error: recv got EOF from 127.0.0.1:51220 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:50:10.383023 24900 catalog_manager.cc:1259] Loaded cluster ID: 55db7512ede64344b96997a3d1e8a1a5
I20260807 19:50:10.383186 24900 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: loading cluster ID for follower catalog manager: success
I20260807 19:50:10.387138 24900 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: acquiring CA information for follower catalog manager: success
I20260807 19:50:10.389364 24900 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43021bd9f5d64e30bffabfa76c77688e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:50:10.418669 24918 catalog_manager.cc:1259] Loaded cluster ID: 55db7512ede64344b96997a3d1e8a1a5
I20260807 19:50:10.418843 24918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: loading cluster ID for follower catalog manager: success
I20260807 19:50:10.423260 24918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: acquiring CA information for follower catalog manager: success
I20260807 19:50:10.426744 24918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 908770844ffc4f3589fe5cd94f506a19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:50:10.435559 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:10.427474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.0.198.253:41637)
0807 19:50:10.427700 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.427709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.428118 (+ 409us) server_negotiation.cc:407] Connection header received
0807 19:50:10.428290 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.428297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.428503 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.428675 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:10.429196 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.430366 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.432155 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.432880 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.433048 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:10.433840 (+ 792us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:10.434659 (+ 819us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:10.434829 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:10.435048 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 19:50:10.435162 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:50:10.450819 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:10.443484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.0.198.252:44181)
0807 19:50:10.443747 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.443757 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.444339 (+ 582us) server_negotiation.cc:407] Connection header received
0807 19:50:10.444556 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.444561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.444717 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.444880 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:10.445386 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.446139 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.448291 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.448755 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.448872 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:10.449527 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:10.450127 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:10.450257 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:10.450397 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 19:50:10.450490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:50:10.470332 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:10.458772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.0.198.254:34765)
0807 19:50:10.459151 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.459157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.459821 (+ 664us) server_negotiation.cc:407] Connection header received
0807 19:50:10.460507 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.460514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.460768 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.460912 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:10.461782 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.463077 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.466040 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.467160 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.467343 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:10.468050 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:10.468752 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:10.468900 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:10.469801 (+ 901us) server_negotiation.cc:299] Negotiation successful
0807 19:50:10.469911 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260807 19:50:10.474764 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:10.458572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.0.198.253:41637)
0807 19:50:10.458977 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.458984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.465874 (+ 6890us) server_negotiation.cc:407] Connection header received
0807 19:50:10.466307 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.466314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.466415 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.466496 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:10.467883 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.469068 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.472329 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.472980 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.473131 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:10.473856 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:10.474032 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:10.474126 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:10.474378 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:50:10.474475 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260807 19:50:10.476635 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24649
I20260807 19:50:10.477857 24714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:10.479758 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:10.460390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.0.198.252:44181)
0807 19:50:10.460626 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:10.460635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:10.470228 (+ 9593us) server_negotiation.cc:407] Connection header received
0807 19:50:10.470774 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:10.470782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:10.470880 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:10.470951 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:10.471844 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.472870 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.477213 (+ 4343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:10.477991 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:10.478250 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:10.478779 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:10.478934 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:10.479032 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:10.479281 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 19:50:10.479413 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:50:10.607584 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24649
I20260807 19:50:10.638792 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24727
I20260807 19:50:10.640283 24793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:10.758265 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24727
I20260807 19:50:10.784744 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 24807
I20260807 19:50:10.786190 24874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:10.907492 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 24807
2026-08-07T19:50:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5118 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:50:11 dist-test-slave-btw8 krb5kdc[25026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:50:11 dist-test-slave-btw8 krb5kdc[25026](info): set up 2 sockets
Aug 07 19:50:11 dist-test-slave-btw8 krb5kdc[25026](info): commencing operation
krb5kdc: starting...
W20260807 19:50:12.178294 795 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/test-user.keytab.
Aug 07 19:50:12 dist-test-slave-btw8 krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:50:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:50:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.198.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
Entry for principal kudu/127.0.198.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab.
I20260807 19:50:12.588224 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34123
--webserver_interface=127.0.198.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:50:12.981174 25041 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:12.981580 25041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:12.981657 25041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:12.981746 25041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:12.989852 25041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:12.989979 25041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:12.990048 25041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:12.990096 25041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:12.990175 25041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:13.000967 25041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.254.keytab
--principal=kudu/127.0.198.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.254:34123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.198.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:13.002431 25041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:13.003794 25041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:13.016957 25046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:13.017103 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:13.017616 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:13.018110 25041 server_base.cc:1034] running on GCE node
I20260807 19:50:13.019204 25041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:13.021361 25041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:13.022748 25041 hybrid_clock.cc:648] HybridClock initialized: now 1786132213022692 us; error 123 us; skew 500 ppm
I20260807 19:50:13.023175 25041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:13.031749 25041 init.cc:377] Logged in from keytab as kudu/127.0.198.254@KRBTEST.COM (short username kudu)
I20260807 19:50:13.033850 25041 webserver.cc:460] Webserver started at http://127.0.198.254:35093/ using document root <none> and password file <none>
I20260807 19:50:13.034776 25041 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:13.034960 25041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:13.035477 25041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:13.039623 25041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data/instance:
uuid: "860fb1359c1e45beb770c0df4585d1d2"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "0dd96f05f32cafaae6a8479b748177b5"
server_key_iv: "d112a953d06af62a2ed1eb8033559feb"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.040630 25041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance:
uuid: "860fb1359c1e45beb770c0df4585d1d2"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "0dd96f05f32cafaae6a8479b748177b5"
server_key_iv: "d112a953d06af62a2ed1eb8033559feb"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.047145 25041 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:50:13.051816 25058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:13.053671 25041 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:50:13.054021 25041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal
uuid: "860fb1359c1e45beb770c0df4585d1d2"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "0dd96f05f32cafaae6a8479b748177b5"
server_key_iv: "d112a953d06af62a2ed1eb8033559feb"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.054311 25041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:13.084136 25041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:13.089136 25041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:13.089532 25041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:13.111061 25041 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.254:34123
I20260807 19:50:13.111090 25115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.254:34123 every 8 connection(s)
I20260807 19:50:13.112823 25041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/data/info.pb
I20260807 19:50:13.117573 25116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:13.120301 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 25041
I20260807 19:50:13.120545 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-0/wal/instance
I20260807 19:50:13.121026 795 external_mini_cluster.cc:1428] Setting key 27f3452fd9068580cc826db15eab5d9f
I20260807 19:50:13.125560 25116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:13.157764 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.129481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34123 (local address 127.0.0.1:44712)
0807 19:50:13.130586 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.130651 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:13.133769 (+ 3118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:13.134956 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:13.134966 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:13.135956 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:13.136546 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.136580 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.138592 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.138597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.139921 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.139936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.141431 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.141437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.141750 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.142669 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:13.142698 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:13.146354 (+ 3656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:13.152356 (+ 6002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.152369 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.152383 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.152987 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.153677 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.153681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.153683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.153879 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.155171 (+ 1292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:13.155179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:13.155530 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 19:50:13.155864 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 19:50:13.156368 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":46,"thread_start_us":425,"threads_started":1}
I20260807 19:50:13.157810 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.130831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.0.198.254:34123)
0807 19:50:13.131574 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:13.131581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:13.131635 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:50:13.134302 (+ 2667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:13.134309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:13.134657 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:13.134860 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:13.136880 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.138378 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.140087 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.141126 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.141363 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.146753 (+ 5390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:13.146771 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:13.146796 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:13.146864 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:13.152107 (+ 5243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.153199 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.153213 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.153220 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.153486 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.154019 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.154027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.154034 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.154937 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:13.155273 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:13.156083 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 19:50:13.156585 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":203,"server-negotiator.queue_time_us":556,"thread_start_us":351,"threads_started":1}
I20260807 19:50:13.161846 25116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:13.163323 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.162681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40224)
0807 19:50:13.162935 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.163103 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
WARNING: no policy specified for kudu/127.0.198.253@KRBTEST.COM; defaulting to no policy
W20260807 19:50:13.163828 25059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
I20260807 19:50:13.165903 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.165387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40226)
0807 19:50:13.165637 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.165727 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 19:50:13.166210 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:34589: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
Entry for principal kudu/127.0.198.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab.
I20260807 19:50:13.215077 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:34589
--webserver_interface=127.0.198.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:50:13.215668 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } attempt: 1
I20260807 19:50:13.217044 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.216484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40236)
0807 19:50:13.216762 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.216898 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 19:50:13.218261 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.217832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40242)
0807 19:50:13.218004 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.218101 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 19:50:13.218479 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:34589: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
I20260807 19:50:13.286783 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } attempt: 2
I20260807 19:50:13.288282 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.287637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40258)
0807 19:50:13.287943 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.288072 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:50:13.289685 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.289182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40264)
0807 19:50:13.289365 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.289487 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:50:13.289951 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:34589: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
I20260807 19:50:13.381295 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } attempt: 3
I20260807 19:50:13.382907 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.382176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40274)
0807 19:50:13.382473 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.382598 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 19:50:13.384891 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.384399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40290)
0807 19:50:13.384609 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.384705 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260807 19:50:13.385270 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:34589: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
I20260807 19:50:13.528606 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } attempt: 4
I20260807 19:50:13.530203 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.529532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40302)
0807 19:50:13.529955 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.530047 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240}
I20260807 19:50:13.531776 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.531176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40304)
0807 19:50:13.531485 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.531610 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 19:50:13.532063 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.253:34589: Network error: Client connection negotiation failed: client connection to 127.0.198.253:34589: connect: Connection refused (error 111)
W20260807 19:50:13.620169 25123 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:13.620597 25123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:13.620730 25123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:13.620837 25123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:50:13.628868 25123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:13.628994 25123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:13.629081 25123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:13.629179 25123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:13.629287 25123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:13.644201 25123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.253.keytab
--principal=kudu/127.0.198.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.253:34589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.198.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:13.645694 25123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:13.647121 25123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:13.660324 25131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:13.660358 25129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:13.660424 25128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:13.662029 25123 server_base.cc:1034] running on GCE node
I20260807 19:50:13.663126 25123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:13.664872 25123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:13.666173 25123 hybrid_clock.cc:648] HybridClock initialized: now 1786132213666045 us; error 94 us; skew 500 ppm
I20260807 19:50:13.666584 25123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:13.670432 25123 init.cc:377] Logged in from keytab as kudu/127.0.198.253@KRBTEST.COM (short username kudu)
I20260807 19:50:13.672542 25123 webserver.cc:460] Webserver started at http://127.0.198.253:40519/ using document root <none> and password file <none>
I20260807 19:50:13.673367 25123 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:13.673516 25123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:13.673837 25123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:13.677282 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data/instance:
uuid: "d890534a98ea4df09d6ae152a2cc7a6d"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "fbd3833273f69377cddc2ef5be3bf174"
server_key_iv: "602d549e09a52e97d74ec02e50f52028"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.678161 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal/instance:
uuid: "d890534a98ea4df09d6ae152a2cc7a6d"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "fbd3833273f69377cddc2ef5be3bf174"
server_key_iv: "602d549e09a52e97d74ec02e50f52028"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.684384 25123 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:50:13.688706 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:13.690770 25123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:50:13.690980 25123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal
uuid: "d890534a98ea4df09d6ae152a2cc7a6d"
format_stamp: "Formatted at 2026-08-07 19:50:13 on dist-test-slave-btw8"
server_key: "fbd3833273f69377cddc2ef5be3bf174"
server_key_iv: "602d549e09a52e97d74ec02e50f52028"
server_key_version: "encryptionkey@0"
I20260807 19:50:13.691215 25123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:13.734148 25123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:13.739276 25123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:13.739641 25123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:13.762640 25123 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.253:34589
I20260807 19:50:13.762652 25194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.253:34589 every 8 connection(s)
I20260807 19:50:13.764370 25123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/data/info.pb
I20260807 19:50:13.768136 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 25123
I20260807 19:50:13.768359 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-1/wal/instance
I20260807 19:50:13.768962 795 external_mini_cluster.cc:1428] Setting key d1f9a91859dcb95de7f604df9411db5e
I20260807 19:50:13.769265 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:13.780139 25195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:13.806839 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.784049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.0.198.254:34123)
0807 19:50:13.784683 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:13.784691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:13.785918 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 19:50:13.788197 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:13.788212 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:13.788326 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:13.788477 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:13.790300 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.791634 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.793378 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.794341 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.794539 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.798916 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:13.798926 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:13.798934 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:13.798991 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:13.803070 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.804206 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.804211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.804213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.804355 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.804848 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.804852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.804854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.805163 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:13.805342 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:13.806143 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 19:50:13.806395 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":327,"threads_started":1}
WARNING: no policy specified for kudu/127.0.198.252@KRBTEST.COM; defaulting to no policy
I20260807 19:50:13.807984 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.784740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34123 (local address 127.0.0.1:44714)
0807 19:50:13.785697 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.785753 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:13.787854 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:13.788763 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:13.788772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:13.789389 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:13.790020 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.790039 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.791841 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.791854 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.793183 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.793197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.794623 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.794628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.794858 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.795678 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:13.795704 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:13.798588 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:13.803309 (+ 4721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.803319 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.803336 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.804021 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.804484 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.804490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.804498 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.804709 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.805299 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:13.805309 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:13.805749 (+ 440us) client_negotiation.cc:769] Sending connection context
0807 19:50:13.806038 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 19:50:13.806649 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":344,"threads_started":1}
I20260807 19:50:13.810240 25195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.198.252@KRBTEST.COM" created.
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:13.827088 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.811196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40316)
0807 19:50:13.811501 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.811524 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:13.811650 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:13.813237 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:13.813242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:13.813730 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:13.813885 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.813895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.815422 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.815428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.816288 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.816300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.817190 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.817195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.817440 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.818104 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:13.818123 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:13.820657 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:13.824370 (+ 3713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.824377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.824382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.824782 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.825185 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.825192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.825195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.825341 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.825993 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:13.825997 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:13.826170 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 19:50:13.826302 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 19:50:13.826616 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":37}
I20260807 19:50:13.827142 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.811479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.0.198.253:34589)
0807 19:50:13.812745 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:13.812751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:13.812781 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:50:13.812831 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:13.812838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:13.813057 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:13.813237 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:13.814074 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.815223 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.816434 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.817025 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.817177 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.820967 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:13.820986 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:13.821002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:13.821040 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:13.824209 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.824941 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.824949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.824951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.825070 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.825451 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.825460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.825462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.825869 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:13.826077 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:13.826466 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 19:50:13.826743 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":1059,"thread_start_us":907,"threads_started":1}
I20260807 19:50:13.830222 25195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:13.831343 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } attempt: 5
I20260807 19:50:13.831748 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.831046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34756)
0807 19:50:13.831284 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.831498 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:50:13.832170 25142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:13.834017 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.833562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34772)
0807 19:50:13.833723 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.833844 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:50:13.835135 25195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Aug 07 19:50:13 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:13.850504 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.833305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.0.198.253:34589)
0807 19:50:13.833534 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:13.833542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:13.833564 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:50:13.833628 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:13.833636 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:13.833743 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:13.833884 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:13.834987 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.836278 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.837597 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.838422 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.838601 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.842136 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:13.842147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:13.842155 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:13.842217 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:13.846827 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.848174 (+ 1347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.848184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.848188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.848386 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:13.848918 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:13.848923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:13.848927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:13.849388 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:13.849538 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:13.849988 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 19:50:13.850240 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:50:13.850580 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.832019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40330)
0807 19:50:13.832247 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.832274 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:13.832423 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:13.834140 (+ 1717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:13.834150 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:13.834564 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:13.834825 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.834835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.836450 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.836455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.837383 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:13.837398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:13.838615 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:13.838620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:13.838812 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:13.839587 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:13.839609 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:13.841866 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:13.847131 (+ 5265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.847138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.847141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.847967 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.848580 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:13.848584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:13.848587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:13.848777 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:13.849552 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:13.849556 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:13.849736 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 19:50:13.849876 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:50:13.850297 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:50:13.851855 25116 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:50:13.853497 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.852933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34780)
0807 19:50:13.853167 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.853287 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:50:13.854832 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.854281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34792)
0807 19:50:13.854586 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.854707 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:50:13.855043 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
Entry for principal kudu/127.0.198.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab.
I20260807 19:50:13.861065 795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
/tmp/dist-test-taskYvfPwA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:34501
--webserver_interface=127.0.198.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.198.212:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:50:13.884533 25195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 1
I20260807 19:50:13.885910 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.885367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34796)
0807 19:50:13.885621 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.885739 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:50:13.886950 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.886588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34800)
0807 19:50:13.886750 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.886838 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:50:13.887171 25195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:13.906317 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 1
I20260807 19:50:13.907703 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.907118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34804)
0807 19:50:13.907394 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.907504 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 19:50:13.908865 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.908480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34812)
0807 19:50:13.908642 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.908738 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:50:13.909197 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:13.955425 25195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 2
I20260807 19:50:13.956861 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.956195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34814)
0807 19:50:13.956457 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.956581 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:50:13.958442 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.957934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34818)
0807 19:50:13.958127 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.958267 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:50:13.958783 25195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:13.977515 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 2
I20260807 19:50:13.978909 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.978310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34820)
0807 19:50:13.978578 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.978681 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:50:13.980229 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:13.979787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34836)
0807 19:50:13.979970 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:13.980062 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:50:13.980535 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:14.050100 25195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 3
I20260807 19:50:14.051607 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.050915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34850)
0807 19:50:14.051193 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.051335 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:50:14.053208 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.052502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34856)
0807 19:50:14.052844 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.052996 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
W20260807 19:50:14.053526 25195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:14.086825 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 3
I20260807 19:50:14.088518 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.087754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34864)
0807 19:50:14.088082 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.088262 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:50:14.089720 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.089288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34876)
0807 19:50:14.089521 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.089600 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260807 19:50:14.089959 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
I20260807 19:50:14.196800 25195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 4
I20260807 19:50:14.198136 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.197521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34882)
0807 19:50:14.197771 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.197891 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:50:14.199432 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.198963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34888)
0807 19:50:14.199133 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.199241 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:50:14.199819 25195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
W20260807 19:50:14.262410 25204 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:50:14.262873 25204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:50:14.262956 25204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:50:14.263043 25204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:50:14.267212 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 4
I20260807 19:50:14.268636 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.268038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34896)
0807 19:50:14.268303 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.268429 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:50:14.269842 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.269411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34912)
0807 19:50:14.269587 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.269684 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:50:14.270097 25116 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.198.252:34501: Network error: Client connection negotiation failed: client connection to 127.0.198.252:34501: connect: Connection refused (error 111)
W20260807 19:50:14.271685 25204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:50:14.271778 25204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:50:14.271914 25204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:50:14.271991 25204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:50:14.272099 25204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:50:14.282811 25204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.198.212:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.198.254:34123,127.0.198.253:34589,127.0.198.252:34501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/krb5kdc/kudu_127.0.198.252.keytab
--principal=kudu/127.0.198.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.198.252:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.198.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:50:14.284332 25204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:50:14.285741 25204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:50:14.298981 25210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:14.298870 25209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:50:14.298915 25212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:50:14.300519 25204 server_base.cc:1034] running on GCE node
I20260807 19:50:14.301438 25204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:50:14.303010 25204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:50:14.304261 25204 hybrid_clock.cc:648] HybridClock initialized: now 1786132214304159 us; error 74 us; skew 500 ppm
I20260807 19:50:14.304600 25204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:50:14.308125 25204 init.cc:377] Logged in from keytab as kudu/127.0.198.252@KRBTEST.COM (short username kudu)
I20260807 19:50:14.309716 25204 webserver.cc:460] Webserver started at http://127.0.198.252:36743/ using document root <none> and password file <none>
I20260807 19:50:14.310463 25204 fs_manager.cc:359] Metadata directory not provided
I20260807 19:50:14.310575 25204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:50:14.310978 25204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:50:14.314427 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data/instance:
uuid: "91489276ba0041acbbcf8a815f2f154f"
format_stamp: "Formatted at 2026-08-07 19:50:14 on dist-test-slave-btw8"
server_key: "4c5db11eb01ba406a9211c8f4d5465e4"
server_key_iv: "e8c1ea5183078bd8c4a64cf79d1af8a0"
server_key_version: "encryptionkey@0"
I20260807 19:50:14.315295 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal/instance:
uuid: "91489276ba0041acbbcf8a815f2f154f"
format_stamp: "Formatted at 2026-08-07 19:50:14 on dist-test-slave-btw8"
server_key: "4c5db11eb01ba406a9211c8f4d5465e4"
server_key_iv: "e8c1ea5183078bd8c4a64cf79d1af8a0"
server_key_version: "encryptionkey@0"
I20260807 19:50:14.321398 25204 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:50:14.325225 25220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:50:14.326823 25204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:50:14.327023 25204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data,/tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal
uuid: "91489276ba0041acbbcf8a815f2f154f"
format_stamp: "Formatted at 2026-08-07 19:50:14 on dist-test-slave-btw8"
server_key: "4c5db11eb01ba406a9211c8f4d5465e4"
server_key_iv: "e8c1ea5183078bd8c4a64cf79d1af8a0"
server_key_version: "encryptionkey@0"
I20260807 19:50:14.327231 25204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:50:14.346493 25204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:50:14.351907 25204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:50:14.352195 25204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:50:14.375013 25204 rpc_server.cc:307] RPC server started. Bound to: 127.0.198.252:34501
I20260807 19:50:14.375032 25276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.198.252:34501 every 8 connection(s)
I20260807 19:50:14.376556 25204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/data/info.pb
I20260807 19:50:14.380712 25277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:50:14.382624 795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu as pid 25204
I20260807 19:50:14.382827 795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYvfPwA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131811070045-795-0/minicluster-data/master-2/wal/instance
I20260807 19:50:14.383391 795 external_mini_cluster.cc:1428] Setting key 66779b349a318e2c830b36a5677e4fce
I20260807 19:50:14.391980 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:14.411319 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.385805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.0.198.252:34501)
0807 19:50:14.386789 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.386803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.386852 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:50:14.388801 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.388832 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.389343 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.389724 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.391838 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.393434 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.395175 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.396218 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.396454 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.401594 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.401607 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.401626 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.401699 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.405748 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.406643 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.406656 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.406668 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.406903 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.407538 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.407544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.407547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.408460 (+ 913us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.408808 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.409345 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.409886 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":434,"threads_started":1}
I20260807 19:50:14.413489 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.395032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.0.198.254:34123)
0807 19:50:14.395685 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.395692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.396555 (+ 863us) server_negotiation.cc:407] Connection header received
0807 19:50:14.396884 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.396891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.396989 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.397131 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.398422 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.399174 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.401111 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.401770 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.401943 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.406123 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.406129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.406133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.406174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.410041 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.410881 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.410888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.410891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.411069 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.411670 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.411675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.411677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.411967 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.412175 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.412773 (+ 598us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.413032 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
I20260807 19:50:14.413516 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.395041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.254:34123 (local address 127.0.0.1:44720)
0807 19:50:14.396306 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.396351 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:14.396669 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:14.397204 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:14.397211 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:14.397898 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:14.398167 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.398176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.399440 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.399445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.400901 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.400922 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.402049 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.402055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.402260 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.403096 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:14.403156 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:14.405855 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:14.410221 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.410232 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.410237 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.410720 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.411219 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.411223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.411226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.411511 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.412120 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:14.412128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:14.412334 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 19:50:14.412533 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 19:50:14.413033 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":776,"threads_started":1}
I20260807 19:50:14.415304 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.254@KRBTEST.COM
I20260807 19:50:14.433400 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.416298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.253:34589 (local address 127.0.0.1:40346)
0807 19:50:14.416503 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.416526 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:14.416640 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:14.417204 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:14.417208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:14.417709 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:14.417868 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.417876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.419262 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.419269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.420740 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.420754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.421773 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.421780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.422006 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.422892 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:14.422923 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:14.426150 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:14.430298 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.430305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.430309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.430881 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.431386 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.431392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.431397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.431569 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.432166 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:14.432171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:14.432459 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 19:50:14.432629 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 19:50:14.432980 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:50:14.433516 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.416270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.0.198.253:34589)
0807 19:50:14.416908 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.416912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.416928 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:50:14.416980 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.416983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.417041 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.417130 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.418076 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.419116 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.420900 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.421564 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.421703 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.426466 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.426473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.426477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.426522 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.430096 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.431022 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.431028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.431032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.431208 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.431713 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.431725 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.431728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.432033 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.432228 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.432781 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.433027 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":413,"threads_started":1}
I20260807 19:50:14.433832 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.415211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.0.198.254:34123)
0807 19:50:14.415426 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.415433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.415767 (+ 334us) server_negotiation.cc:407] Connection header received
0807 19:50:14.415842 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.415848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.415943 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.416056 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.417659 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.418714 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.420733 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.421448 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.421703 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.427429 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.427435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.427438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.427478 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.430301 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.431139 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.431143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.431146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.431445 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.431896 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.431901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.431904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.432210 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.432586 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.432953 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.433216 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 19:50:14.434814 25277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.252@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.198.253@KRBTEST.COM
I20260807 19:50:14.451269 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.435693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.0.198.252:34501)
0807 19:50:14.435870 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.435886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.436154 (+ 268us) server_negotiation.cc:407] Connection header received
0807 19:50:14.436339 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.436343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.436407 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.436510 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.437546 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.438330 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.440013 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.440435 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.440550 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.444623 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.444628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.444631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.444676 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.447899 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.448657 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.448664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.448668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.448850 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.449367 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.449373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.449378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.449811 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.449987 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.450442 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.450714 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:50:14.451390 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.435814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34936)
0807 19:50:14.436060 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.436080 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:14.436206 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:14.436594 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:14.436599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:14.437174 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:14.437391 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.437406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.438480 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.438484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.439866 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.439882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.440621 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.440631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.440833 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.441526 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:14.441554 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:14.444406 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:14.448085 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.448093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.448099 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.448500 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.449032 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.449038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.449042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.449214 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.449982 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:14.449987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:14.450166 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 19:50:14.450336 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 19:50:14.450948 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":217}
I20260807 19:50:14.453620 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.435863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.0.198.253:34589)
0807 19:50:14.436022 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.436025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.436149 (+ 124us) server_negotiation.cc:407] Connection header received
0807 19:50:14.436322 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.436327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.436407 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.436510 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.438194 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.438996 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.440711 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.441205 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.441347 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.446421 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.446429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.446432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.446476 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.449793 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.450812 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.450819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.450822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.451041 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.451774 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.451779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.451783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.452262 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.452577 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.453078 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.453333 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:50:14.458410 25277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: Bootstrap starting.
I20260807 19:50:14.459770 795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:50:14.462620 25277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:14.463889 25277 log.cc:824] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:14.462627 25088 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "860fb1359c1e45beb770c0df4585d1d2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44726
I20260807 19:50:14.466975 25277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: No bootstrap required, opened a new log
I20260807 19:50:14.474797 25277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.475248 25277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91489276ba0041acbbcf8a815f2f154f, State: Initialized, Role: FOLLOWER
I20260807 19:50:14.475785 25277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.477030 25287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.477530 25287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.478384 25277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:50:14.490509 25299 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:50:14.490695 25299 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:50:14.488104 25204 master_runner.cc:186] Error getting master registration for 127.0.198.254:34123: OK, instance_id { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" instance_seqno: 1786132213104798 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:50:14.498327 25204 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:50:14.499081 25195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 5
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.253@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
I20260807 19:50:14.518878 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.499922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34950)
0807 19:50:14.500126 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.500147 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:14.500273 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:14.500841 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:14.500846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:14.501678 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:14.501936 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.501948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.503149 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.503155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.504147 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.504161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.505036 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.505044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.505309 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.506036 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:14.506065 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:14.509565 (+ 3500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:14.515257 (+ 5692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.515265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.515270 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.516014 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.516602 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.516608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.516613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.516814 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.517701 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:14.517707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:14.517897 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 19:50:14.518077 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 19:50:14.518458 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:50:14.518960 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.500004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.0.198.252:34501)
0807 19:50:14.500234 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.500241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.500266 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:50:14.500432 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.500444 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.500650 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.500806 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.502152 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.502971 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.504312 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.504851 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.505000 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.509904 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.509913 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.509922 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.509970 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.514979 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.516209 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.516216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.516220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.516445 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.516953 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.516958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.516965 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.517547 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.517716 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.518290 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.518583 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:50:14.524432 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: Bootstrap starting.
I20260807 19:50:14.528990 25195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:14.530470 25195 log.cc:824] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:14.534606 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: No bootstrap required, opened a new log
I20260807 19:50:14.546821 25195 raft_consensus.cc:348] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.547403 25195 raft_consensus.cc:729] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d890534a98ea4df09d6ae152a2cc7a6d, State: Initialized, Role: FOLLOWER
I20260807 19:50:14.547412 25116 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } attempt: 5
I20260807 19:50:14.548050 25195 consensus_queue.cc:260] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.550334 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.551467 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.551925 25195 sys_catalog.cc:564] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 19:50:14 dist-test-slave-btw8 krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.198.254@KRBTEST.COM for kudu/127.0.198.252@KRBTEST.COM
W20260807 19:50:14.576090 25315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:50:14.576278 25315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:50:14.578751 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.548415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.198.252:34501 (local address 127.0.0.1:34966)
0807 19:50:14.548689 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:50:14.548717 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:50:14.548909 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:50:14.549998 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:50:14.550005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:50:14.550599 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:50:14.550852 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.550869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.552683 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.552692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.554540 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:50:14.554578 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.556091 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.556099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:50:14.556425 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.557241 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:50:14.557269 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:50:14.560599 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:50:14.567331 (+ 6732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.567337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.567344 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.568026 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.572281 (+ 4255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:50:14.572289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:50:14.572297 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:50:14.572545 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:50:14.577390 (+ 4845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:50:14.577398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:50:14.577626 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 19:50:14.577817 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 19:50:14.578220 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:50:14.580410 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.549143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.0.198.252:34501)
0807 19:50:14.549412 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.549420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.549443 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:50:14.549520 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.549526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.549619 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.549742 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.551149 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.552443 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.554861 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.555835 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.556021 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.560952 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.560964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.560969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.561033 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.567138 (+ 6105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.571783 (+ 4645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.571793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.571799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.572071 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.576467 (+ 4396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.576475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.576479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.576955 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.579471 (+ 2516us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.579676 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.579996 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:50:14.567435 25123 master_runner.cc:186] Error getting master registration for 127.0.198.254:34123: OK, instance_id { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" instance_seqno: 1786132213104798 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:50:14.586611 25123 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:50:14.587713 25116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2: Bootstrap starting.
I20260807 19:50:14.592664 25116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2: Neither blocks nor log segments found. Creating new log.
I20260807 19:50:14.594606 25116 log.cc:824] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:50:14.599598 25116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2: No bootstrap required, opened a new log
W20260807 19:50:14.604664 25227 tablet.cc:2369] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:50:14.608300 25116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.608805 25116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860fb1359c1e45beb770c0df4585d1d2, State: Initialized, Role: FOLLOWER
I20260807 19:50:14.609417 25116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.610989 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.612071 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.612699 25116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:50:14.612761 25088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:14.613001 25088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:14.617977 25088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.621238 25167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "860fb1359c1e45beb770c0df4585d1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d890534a98ea4df09d6ae152a2cc7a6d"
I20260807 19:50:14.621940 25167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:14.626550 25167 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860fb1359c1e45beb770c0df4585d1d2 in term 1.
I20260807 19:50:14.627693 25059 leader_election.cc:302] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 860fb1359c1e45beb770c0df4585d1d2, d890534a98ea4df09d6ae152a2cc7a6d; no voters:
I20260807 19:50:14.629868 25251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "860fb1359c1e45beb770c0df4585d1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "91489276ba0041acbbcf8a815f2f154f"
I20260807 19:50:14.630507 25251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:50:14.634740 25251 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860fb1359c1e45beb770c0df4585d1d2 in term 1.
I20260807 19:50:14.636183 25088 leader_election.cc:288] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [CANDIDATE]: Term 1 election: Requested vote from peers d890534a98ea4df09d6ae152a2cc7a6d (127.0.198.253:34589), 91489276ba0041acbbcf8a815f2f154f (127.0.198.252:34501)
I20260807 19:50:14.643591 25318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:50:14.653301 25318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [term 1 LEADER]: Becoming Leader. State: Replica: 860fb1359c1e45beb770c0df4585d1d2, State: Running, Role: LEADER
I20260807 19:50:14.654821 25318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } }
I20260807 19:50:14.667518 25323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860fb1359c1e45beb770c0df4585d1d2. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.668200 25323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:14.671986 25335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:50:14.702971 25335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:50:14.741063 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.677471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.0.198.254:34123)
0807 19:50:14.677768 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.677776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.693751 (+ 15975us) server_negotiation.cc:407] Connection header received
0807 19:50:14.693966 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.693973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.694099 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.694226 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.703857 (+ 9631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.705399 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.720839 (+ 15440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.721614 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.721737 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.725941 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.725951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.725957 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.726005 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.730064 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.731147 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.731153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.731156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.731375 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.731981 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.731987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.731991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.732476 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.732859 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.740265 (+ 7406us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.740635 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:50:14.742620 25146 tablet.cc:2369] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:50:14.753300 25167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [term 1 FOLLOWER]: Refusing update from remote peer 860fb1359c1e45beb770c0df4585d1d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:50:14.754817 25323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:50:14.751269 25251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [term 1 FOLLOWER]: Refusing update from remote peer 860fb1359c1e45beb770c0df4585d1d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:50:14.762521 25323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:50:14.785411 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.678679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.0.198.252:34501)
0807 19:50:14.678916 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.678922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.695850 (+ 16928us) server_negotiation.cc:407] Connection header received
0807 19:50:14.696130 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.696136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.696243 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.696337 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.705850 (+ 9513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.707267 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.716797 (+ 9530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.717596 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.717765 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.749015 (+ 31250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.749024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.749030 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.749091 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.767688 (+ 18597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.776527 (+ 8839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.776533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.776537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.776738 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.780058 (+ 3320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.780066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.780071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.780549 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.784535 (+ 3986us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.784708 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.784985 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:50:14.786809 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:14.676443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.0.198.253:34589)
0807 19:50:14.676636 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:14.676644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:14.690018 (+ 13374us) server_negotiation.cc:407] Connection header received
0807 19:50:14.690381 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:14.690388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:14.690485 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:14.690562 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:14.709586 (+ 19024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.710867 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.715592 (+ 4725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:14.716358 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:14.716534 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:14.740774 (+ 24240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:50:14.740783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:50:14.740788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:50:14.740853 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:50:14.771108 (+ 30255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.774831 (+ 3723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.774839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.774845 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.775042 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:50:14.778733 (+ 3691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:50:14.778743 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:50:14.778751 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:50:14.779259 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:50:14.782270 (+ 3011us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:14.782448 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 19:50:14.782754 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:50:14.816905 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860fb1359c1e45beb770c0df4585d1d2. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.817268 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.840694 25346 mvcc.cc:205] Tried to move back new op lower bound from 7315997551594733568 to 7315997551251501056. Current Snapshot: MvccSnapshot[applied={T|T < 7315997551594733568}]
I20260807 19:50:14.847644 25287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860fb1359c1e45beb770c0df4585d1d2. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.848032 25287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.860400 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.860749 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.873353 25335 catalog_manager.cc:1347] Generated new cluster ID: 922effbf5e534e1aa2b897bd5fb454a5
I20260807 19:50:14.873494 25335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:50:14.875638 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.875979 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:14.876204 25345 mvcc.cc:205] Tried to move back new op lower bound from 7315997551594733568 to 7315997551251501056. Current Snapshot: MvccSnapshot[applied={T|T < 7315997551594733568}]
I20260807 19:50:14.876271 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.876541 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:50:14.897445 25287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "860fb1359c1e45beb770c0df4585d1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860fb1359c1e45beb770c0df4585d1d2" member_type: VOTER last_known_addr { host: "127.0.198.254" port: 34123 } } peers { permanent_uuid: "d890534a98ea4df09d6ae152a2cc7a6d" member_type: VOTER last_known_addr { host: "127.0.198.253" port: 34589 } } peers { permanent_uuid: "91489276ba0041acbbcf8a815f2f154f" member_type: VOTER last_known_addr { host: "127.0.198.252" port: 34501 } } }
I20260807 19:50:14.897764 25287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:50:14.957432 25335 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:50:14.959805 25335 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:50:15.006387 25335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 860fb1359c1e45beb770c0df4585d1d2: Generated new TSK 0
I20260807 19:50:15.007709 25335 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:50:15.175104 25282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.162801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.0.198.254:34123)
0807 19:50:15.163093 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.163103 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.172138 (+ 9035us) server_negotiation.cc:407] Connection header received
0807 19:50:15.172598 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.172608 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.172751 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.172866 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.174730 (+ 1864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40562: BlockingRecv error: recv got EOF from 127.0.0.1:40562 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:50:15.178153 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.164299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59936 (local address 127.0.198.253:34589)
0807 19:50:15.164561 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.164574 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.176174 (+ 11600us) server_negotiation.cc:407] Connection header received
0807 19:50:15.176489 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.176495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.176592 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.176695 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.177861 (+ 1166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59936: BlockingRecv error: recv got EOF from 127.0.0.1:59936 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:50:15.178701 25281 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.173303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.0.198.252:34501)
0807 19:50:15.173537 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.173545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.176302 (+ 2757us) server_negotiation.cc:407] Connection header received
0807 19:50:15.176989 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.176995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.177098 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.177192 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.178444 (+ 1252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50102: BlockingRecv error: recv got EOF from 127.0.0.1:50102 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 19:50:15.211782 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.209024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.0.198.253:34589)
0807 19:50:15.209284 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.209290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.209867 (+ 577us) server_negotiation.cc:407] Connection header received
0807 19:50:15.210139 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.210145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.210285 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.210416 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.211478 (+ 1062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59946: BlockingRecv error: recv got EOF from 127.0.0.1:59946 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:50:15.224053 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.221276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.0.198.253:34589)
0807 19:50:15.221493 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.221499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.222081 (+ 582us) server_negotiation.cc:407] Connection header received
0807 19:50:15.222338 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.222348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.222472 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.222627 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.223744 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59958: BlockingRecv error: recv got EOF from 127.0.0.1:59958 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:50:15.232656 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.230098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.0.198.253:34589)
0807 19:50:15.230315 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.230323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.230955 (+ 632us) server_negotiation.cc:407] Connection header received
0807 19:50:15.231201 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.231207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.231317 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.231442 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.232389 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59964: BlockingRecv error: recv got EOF from 127.0.0.1:59964 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260807 19:50:15.243332 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.240669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.0.198.253:34589)
0807 19:50:15.240937 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.240944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.241538 (+ 594us) server_negotiation.cc:407] Connection header received
0807 19:50:15.241798 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.241805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.241931 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.242062 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.243044 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59968: BlockingRecv error: recv got EOF from 127.0.0.1:59968 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
W20260807 19:50:15.258477 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.255768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59978 (local address 127.0.198.253:34589)
0807 19:50:15.255974 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.255980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.256621 (+ 641us) server_negotiation.cc:407] Connection header received
0807 19:50:15.256832 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.256838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.257035 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.257160 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.258183 (+ 1023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59978: BlockingRecv error: recv got EOF from 127.0.0.1:59978 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:50:15.281589 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.278934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.0.198.253:34589)
0807 19:50:15.279237 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.279247 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.279898 (+ 651us) server_negotiation.cc:407] Connection header received
0807 19:50:15.280194 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.280201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.280294 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.280417 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.281313 (+ 896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59994: BlockingRecv error: recv got EOF from 127.0.0.1:59994 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 19:50:15.321020 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.318198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.0.198.253:34589)
0807 19:50:15.318485 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.318497 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.319165 (+ 668us) server_negotiation.cc:407] Connection header received
0807 19:50:15.319442 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.319448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.319590 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.319705 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.320740 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60006: BlockingRecv error: recv got EOF from 127.0.0.1:60006 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:50:15.391929 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.389546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.0.198.253:34589)
0807 19:50:15.389893 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.389900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.390195 (+ 295us) server_negotiation.cc:407] Connection header received
0807 19:50:15.390390 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.390394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.390491 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.390594 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.391675 (+ 1081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60016: BlockingRecv error: recv got EOF from 127.0.0.1:60016 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 19:50:15.492901 25299 catalog_manager.cc:1259] Loaded cluster ID: 922effbf5e534e1aa2b897bd5fb454a5
I20260807 19:50:15.493021 25299 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: loading cluster ID for follower catalog manager: success
I20260807 19:50:15.496416 25299 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: acquiring CA information for follower catalog manager: success
I20260807 19:50:15.498798 25299 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91489276ba0041acbbcf8a815f2f154f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 19:50:15.526258 25284 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:50:15.524202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.0.198.253:34589)
0807 19:50:15.524411 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.524417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.524784 (+ 367us) server_negotiation.cc:407] Connection header received
0807 19:50:15.524959 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.524964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.525066 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.525193 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:50:15.526020 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60032: BlockingRecv error: recv got EOF from 127.0.0.1:60032 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:50:15.579030 25315 catalog_manager.cc:1259] Loaded cluster ID: 922effbf5e534e1aa2b897bd5fb454a5
I20260807 19:50:15.579150 25315 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: loading cluster ID for follower catalog manager: success
I20260807 19:50:15.581912 25315 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: acquiring CA information for follower catalog manager: success
I20260807 19:50:15.584048 25315 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d890534a98ea4df09d6ae152a2cc7a6d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:50:15.796391 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:15.789283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.0.198.253:34589)
0807 19:50:15.789593 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.789602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.790061 (+ 459us) server_negotiation.cc:407] Connection header received
0807 19:50:15.790355 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.790360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.790533 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.790641 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:15.791203 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.791986 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.793871 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.794315 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.794470 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:15.795022 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:15.795724 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:15.795892 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:15.796000 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 19:50:15.796098 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260807 19:50:15.814800 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:15.803500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.0.198.252:34501)
0807 19:50:15.807877 (+ 4377us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.807887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.807919 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:50:15.808106 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.808115 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.808383 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.808629 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:15.809349 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.810205 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.811901 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.812308 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.812414 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:15.813139 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:15.813870 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:15.813987 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:15.814211 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:50:15.814421 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4196,"thread_start_us":4002,"threads_started":1}
I20260807 19:50:15.837934 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:15.822510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.0.198.252:34501)
0807 19:50:15.828311 (+ 5801us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.828324 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.830224 (+ 1900us) server_negotiation.cc:407] Connection header received
0807 19:50:15.830530 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.830540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.830690 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.830829 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:15.831775 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.832985 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.835536 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.836062 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.836235 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:15.836935 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:15.837105 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:15.837238 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:15.837454 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 19:50:15.837582 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5492}
I20260807 19:50:15.839799 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:15.822269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.0.198.254:34123)
0807 19:50:15.829254 (+ 6985us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.829266 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.831261 (+ 1995us) server_negotiation.cc:407] Connection header received
0807 19:50:15.831540 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.831548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.831815 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.832118 (+ 303us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:15.832688 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.834039 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.836705 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.837370 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.837518 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:15.838397 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:15.838959 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:15.839098 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:15.839267 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 19:50:15.839426 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6756,"thread_start_us":6552,"threads_started":1}
I20260807 19:50:15.840305 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:50:15.822355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.0.198.253:34589)
0807 19:50:15.822599 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:50:15.822608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:50:15.832162 (+ 9554us) server_negotiation.cc:407] Connection header received
0807 19:50:15.832462 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:50:15.832469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:50:15.832575 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:50:15.832676 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:50:15.833575 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.834874 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.837753 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:50:15.838526 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:50:15.838675 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:50:15.839397 (+ 722us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:50:15.839545 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:50:15.839621 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:50:15.839849 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:50:15.839943 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 19:50:15.846123 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 25041
I20260807 19:50:15.847679 25111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:15.976612 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 25041
I20260807 19:50:16.006017 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 25123
I20260807 19:50:16.007452 25189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:16.129796 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 25123
I20260807 19:50:16.158766 795 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu pid 25204
I20260807 19:50:16.160104 25272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:50:16.281055 795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYvfPwA/build/asan/bin/kudu with pid 25204
2026-08-07T19:50:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5371 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10490 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (405167 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST